My grandfather played poker with a core group of friends (to my knowlegde he did not play in card rooms or casinos) and taught me the basics of the game. We played Five-card draw with penny antes. Those instances and the occasional occasional video poker machine summed up my poker experience until the recent Texas Hold’em craze that started with Chris Moneymaker‘s 2003 World Series of Poker victory and the explosion of online gambling. I learned the ropes of that poker variant and play with friends and in casinos when the opportunity arises.

A few years ago my grandmother asked me if I wanted any of my grandfather’s belongings (he had passed away several years earlier). I remembered a framed royal flush, signed and dated by witnesses (I’m not sure whether he was playing Five-card draw at the time, and if so how many of the royal flush cards he was dealt vs. exchanged).

On February 28th I was in South Lake Tahoe for a company ski trip and sat down in the Harvey’s poker room with a couple of co-workers (no-limit Texas Hold’em, $2 small blind, $3 big blind). After playing for about an hour, our table of ten started to lose people, eventually becoming a table of five and disbanding. My co-workers moved to a table with two empty seats and I to a table with one (maximum of ten players per table).

After a few unworthy hands the dealer tossed KQ♠ (king and queen of spades) as my starting hand. I sat in fifth position. One person called and one folded in front of me, I called, two people called after me and both blinds joined in. With six players and $18 in the pot, the dealer flopped A♠K♦J♠. The blinds checked, the player in front of me bet $10. I called, the two players behind me called and the blinds folded. Four players left, $53 in the pot (dealer took $4 for the casino whenever the pot reached $40). The turn was 3♣. The person in front of me checked, I checked, the next person checked and then the last player bet $15. The player in front of me folded, I called and the player after me called. Three players left, $97 in the pot. The river was T♠. Holy cow, not only did I have a flush, but I had a royal flush, the highest possible hand in all of poker.

First to act and trying to keep my cool, I took a moment to consider my options. I could check, hoping one of the other two players would bet and then raise them. I could make a small bet, say $20-$30 and perhaps someone would call or re-raise me. Or I could make a sizable bet and it’s possible someone might think I was trying to buy the pot. The people remaining each had over $500 worth of chips in front of them and I had a little less than $200. I put out $75, about 75% of the pot and 40% of my stack in the hopes that someone else had hit something and thought I was attempting to buy the pot. The next guy thought for a while, but eventually folded and the other player quickly folded, oh well. I didn’t have to show my cards, but how could I not show a royal flush?

So I flipped over my cards and the table erupted. It also turned out the casino had a bonus jackpot which grew slowly based on the number of hands dealt and I received 5% of it for showing a royal flush. It wasn’t a boatload of money i.e. I learned the casino isn’t required to report payouts of under $600 to the government, but it was a pleasant surprise. I tipped the dealer about 10% of my bonus and then thought of my grandfather’s royal flush and asked if I could keep the deck of cards. The dealer quickly said that wasn’t allowed, gathered the cards and placed them in the shuffler as if nothing special had happened (he mentioned he had dealt a royal flush earlier in the day). I wanted my tip back.

I received some crappy cards for the next hand and immediately mucked them. As I finished stacking my winnings and lamented not having an item of memorabilia to go with my grandfather’s, not even a photo, something strange happend. The table erupted again as another player had just turned over a royal flush in hearts. The board had AKJT♥ (not sure about the fifth card) on it and they had Q♥ in their hand. This was a first for everyone on the table, including the dealer, two royal flushes in a row. People jokingly asked if the dealer was worried about getting fired, after all he had caused the casino to payout ~15% of the bonus today (he answered no).

My co-workers and I started wondering how often one could expect two royal flushes in a row during a game of Texas Hold’em. In the simplest scenario i.e. choosing 5 random cards from a standard 52 card deck, the odds of getting a royal flush are 1 in 649,350 hands. That number is generated via a branch of mathematics called Combinatorics. First one calculates the number of ways to select 5 cards from 52 (without replacing any during the selection process). There are 52 different ways 1 card can be picked from a deck, one for each card. That leaves 51 cards from which one can select another card, and after that card has been picked, 50 cards remain etc. So one might think the number of ways to select 5 cards from a 52 card deck is 52*51*50*49*48 = 311,875,200.

That calculation would be correct if one was looking for all the permutations of 5 card poker hands i.e. the order was important e.g. selecting A2358♣ was considered different from picking 35A82♣. In poker, combinations not permutations matter. Our original computation can be written using factorial notation i.e. 52!/(52-5)! = 52!/47!. It’s easy to work out with smaller numbers e.g. select 3 cards from a deck of 5 = 5*4*3 = 60 and 5!/(5-3)! = 5!/2! = (5*4*3*2*1)/2*1 = 60. In the case of choosing 3 cards from a deck of 5, one must also divide by 3! since any permutation works. As an example, for 3 items A, B and C there are 6 combinations, ABC, ACB, BAC, BCA, CAB, CBA . This is because there are X different items which can occupy the first position, X-1 for the next, etc. which leads to X!.

Getting back to the 5 cards out of 52 example our the number of possible 5 card hands in poker is 52!/(5!*(52-47)!) or 2,598,560. This is often written as (52 choose 5) and the math becomes N!/(K!*(N-K)!) for any (N choose K). There are 4 possible royal flushes so the odds of getting dealt one given 5 cards are 4 in 2,598,560 or 1 in 649,350. Given 7 cards as in Texas Hold’em, getting a royal flush of 5 cards should happen more often. The number of possible 7 card hands is (52 choose 7) = 133,784,560. If someone has a 5 card ace high straight flush, it doesn’t matter what the other 2 cards are, so we can simply multiply the number of different royal flushes, 4, by the number of ways to choose 2 cards out of the remaining 47 (52-5 since the 5 cards that make up a royal flush are already out of the deck). (47 choose 2) = 1,081, 4*1,081 = 4,324 making the odds 4,324 in 133,784,568 or 1 in 30,940.

At my poker playing peak, probably sometime in 2004 or 2005, I played online a few hours per week, at a friends house a few hours once a month and at a casino for a few hours once a year, so 3*52+3*12+3 = 195 hours of poker/year. If I saw 20 hands/hour, that’d be 20*195 = 3900 hands/year and I so perhaps I’d expect to get one royal flush roughly every 8 years.

But I hadn’t even witnessed a royal flush before; when playing at a table with multiple people, there is more than one 7 card hand happening at a time (often there are 8-10 players). So how often could I expect to witness a royal flush? One of my co-workers came up with this line of thinking. Royal flushes are unique in that no matter how many people are playing, only one person can have a royal flush per hand i.e. the probabilities for each additional player at the table getting a royal flush are mostly disjoint (mostly because if the royal flush appears in the 5 community cards, everyone at the table has a royal flush so that event should only be counted once). We’ve already calculated the odds of getting a royal flush given 7 cards, so for each additional player at the table (assuming they stay in to the river) the probability of witnessing a royal flush goes up by the probability of a single person getting a royal flush. Let’s set some variables:

• RF5 = p(royal flush|5 cards) = probability of a royal flush given 5 cards = 4*(52 choose 5) = 0.00000154
• RF7 = p(royal flush|7 cards) = probability of a royal flush given 7 cards = 4*(47 choose 2)/(52 choose 7) = 0.00003232
• RFX = p(royal flush|X players at the river) = probability of a royal flush with X players at the river = X*RF7-(X-1)*RF5

For RFX the (X-1)*RF5 factor removes the over-counting when the community cards contain a royal flush. The probability of that event is RF5 and it occurs once in each RF7. So subtracting X-1 of them leaves one for everyone to share. Plugging in the numbers for X = 10 yields 0.00030934 or 1 in 3,233 hands.

In reality, the odds are much lower as ten people staying in for the river card at the same table does not happen often, if ever. In many cases, the hand is over before the turn or river card gets shown and if the river does come into play, only a few people remain in the hand. So if on average, one out of every three hands goes to the river and when that happens, on average three players remain RFX is more like RF7-(3-1)*RF5 = 0.00002924 or 1 in 34,200 hands.

My other co-worker attacked the problem in a slightly different way, by summing the three probabilities of various ways a royal flush can occur in a game of Texas Hold’em i.e. 3 of the 5 community cards and both of cards from a single player make a royal flush, 4 of 5 community cards and one card from a single player make a royal flush and when 5 of 5 community cards and 0 cards from every player make a royal flush. To compute the odds that way, let’s set some variables (these again all presume all players at a table stay in until the river every game):

• C1 = p(1 royal flush card|royal flush, X players) = probability of a player getting the 1 card necessary to complete a royal flush given a royal flush is happening and X players are at the table = 2*X/(47 choose 1)
• C2 = p(2 RF cards|royal flush, X players) = probability of a player getting the 2 cards necessary to complete a royal flush given a royal flush is happening and X players are at the table = X/(47 choose 2)
• C3 = p(3 of 5|royal flush) = probability 3 of the 5 community cards are used by a player given a royal flush is happening = ((5 choose 3)*(47 choose 2))/(52 choose 5)
• C4 = p(4 of 5|royal flush) = probability 4 of the 5 community cards are used by a player given a royal flush is happening = ((5 choose 4)*(47 choose 1)/(52 choose 5)
• C5 = p(5 of 5|royal flush) = probability 5 of the 5 community cards are used by a player given a royal flush is happening = (5 choose 5)/(52 choose 5)
• RFX = p(royal flush|X players at the river) = probability of a royal flush with X players at the river = 4*((C1*C4)+(C2*C3)+C5)

A few notes about these variables:

• C1 and C2 presume a royal flush is possible i.e. the community cards have the 4 or 3 necessary cards and just give the odds of a player having the remaining 1 or 2 cards. That’s why 47 shows up in those equations, because 5 of the 52 cards in the deck are already accounted for as the community cards.
• C1 has 2 as a factor in the numerator because a player has 2 chances of getting the remaining 1 necessary card. In C2, the player must have both cards necessary, so there is no extra factor.
• In RFX 4 is a factor because there are 4 possible royal flushes.

Plugging in the numbers for X = 10 we get:

• C1*C4 = (2*10*(5 choose 4)*(47 choose 1))/((47 choose 1)*(52 choose 5)) = (20*(5 choose 4))/(52 choose 5) = 100/2,598,560 = 0.0000385
• C2*C3 = (10*(5 choose 3)*(47 choose 2))/((47 choose 2)*(52 choose 5)) = 10*10/2,598,560 = 0.0000385
• C5 = 1/2,598,560 = 0.000000385
• RFX = 4*(0.0000385+0.0000385+0.000000385) = 0.00030954 = 1 in 3,233 hands

That’s exactly the same result via the first formula, phew. One non-obvious tidbit this way of thinking highlights is the probability of getting a royal flush with 4 cards in the community and one card in your hand (C1*C4) is exactly the same as getting a royal flush with 3 cards in the community and 2 cards in your hand (C2*C3).

Now that all the hard work is done, estimating how often one might expect two royals flushes at a ten player table where everyone stays in until the river on every had would be 0.00030954^2 or 0.00000000958 or 1 in every 10,436,778 hands. If I played 3,900 hands/year as originally estimated, I could expect this event to happen once every 2,676 years. Using our more realistic earlier estimate that one out of every three hands sees a river card and when that happens, there are three players still in the game, a royal flush would happen 1 in 34,200 hands (0.00002924) and a back-to-back occurrence would be 0.000000000855 or 1 in every 1,169,590,643 hands. For some perspective, the odds of winning the MEGA Millions Lottery jackpot are almost 7 times better, approximately 1 in 176,000,000!

Digg did not allow embedding images directly into the post so I got to see the explosion of views firsthand. Recently I noticed a couple of referrers from tumblr.com in my stats, and they all happend to be for the same image of Yogurt Drinks from Amsterdam.

The Flickr stats showed only a few tens of people looked at the image directly on Flickr, but I was still curious what people were posting about my image. It turns out, there wasn’t much. The image was just one in a stream on various image centric tumblelogs. However, there were 775 “notes”, most of which appeared to be in a particular format: TumblrUser this

For example:
fairyfloss89 reblogged this from fusels
jessicajmaciel liked this
jessicajmaciel reblogged this from urban-princess

The UNIX utilities cut, sort, uniq and grep are my friends, so it was relatively easy to get some stats out of this data:

cut -d\ -f2 notes | sort | uniq -c

 156 liked
   1 love
   1 nomk
   1 posted
 606 reblogged
   1 reeeaally
   2 want

That command says using as a delimiter (-d\ ) print the 2nd field (-f2) of each line, sort them and then count (-c) the number of unique occurances. Looks like there were 156 likes and 606 reblogs, so what where the others?

grep -v “liked\|reblogged” notes

i want a smoothie:(
id love to scoop one of these up
omk nomk nomk 
I reeeaally want a mango pineapple smoothie right now.
DRINKING ONE RIGHT NOW
I want them all.
candystaples posted this

That command says print all lines that don’t contain (-v) the words liked or (\|) reblogged. Looks like 6 people decided to post a comment and the source post came from candystaples (here’s a link to the actual post). People can both like and reblog the image, how many of those were there?

cut -d\ -f1 notes | sort | uniq -c | grep “^ \+2″ | wc -l

38

That command says using as the delimiter (-d\ ) print the 1st field (-f1) of each line, sort them, count (-c) the number of unique occurrences, print the lines that begin (^) with 1 or more characters followed by a 2, and count those lines (-l).

So at least 606 + 156 – 38 = 724 unique people saw my image and liked or reblogged it. It turns out, when reblogging the comment format is TumblrUserA reblogged this from TumblrUserB. So candystaples was the source, but which other tumblelogs generated the more reblogs?

grep “reblogged this from” notes | cut -d\ -f5 | sort | uniq -c | sort -r | head”

  86 fusels
  75 iamfuckingglamorous
  37 dingyfeathers
  31 fashionistha
  17 madamelulu
  16 s-o-u-l
  14 urban-princess
  11 breakingthehabits
  10 izzyparadise
   9 sellyourseconds

That command says print all the lines that contain “reblogged this from”, using as a delimiter (-d\ ) print the 5th field (-f5) of each line, sort them, count (-c) the number of unique occurances, sort those in reverse/descending (-r) order and print the first 10 (default for the head command).

I hadn’t really thought about what sort of multiplier to attach to my Flickr stats in order to determine how many people had actually viewed one of my photos. Clearly not every photo deserves a 10x or greater multiplier and in cases like this, 10x is likely to small e.g. if 700 people took the time to perform a like or reblog action, is that 1% of the number of people who actually saw the image? And this doesn’t take into account the times photos are just copied and posted elsewhere with no attribution e.g. my experience with a Famous Board Games Photo.

The next morning we were at the car by 8:45a and to my surprise there was a parking ticket on the windshield. I looked around and saw a pay for parking kiosk about 75ft behind the car which I had missed the previous night as I had not looked in that direction. I vowed to contest it when I got home and wrote this:

December 4, 2011

City of Sacramento, Revenue Services
P.O. Box 2551
Sacramento CA 95812-2551

Dear Sir/Madam,

I would like to contest this parking citation:

Citation #: #########
Issue Date: 25-Nov-11
Location: 1215 J St
Violation Code: 10.20.090 Meter Expired
Vehicle License #: #######

I parked at the above location around 10pm on 24-Nov-11 and saw the sign that said 1hr parking 8am-6pm. The next morning at 8:45am I went to move my car and saw I had been issued a parking ticket. There was no meter on the sidewalk next to the spot and no sign indicating the spot was metered. Here is a photo I took that morning from my car of the parking sign as viewed from the driver’s seat. I appreciate your time and understanding.

Sincerely,

James Pine

Because this was a legal challenge, I sent the letter via certified mail and requested a return receipt. My total postage cost was $5.74 or 13% of the actual fine! After nearly 3 months, I received the judgement and of course they denied my request to waive the fee, citing rule #2 of the 13 Ways to Park Legally in Sacramento. Looking 100ft in either direction due to these parking pay stations seems like a step backward from the time when a meter right next to your car contained all the information necessary. To be fair, accepting credit cards and paper money in addition to coins is a step forward…though the number of times a parking machine has failed to accept my credit card or paper money is non-zero. Anyway, the real problem was that I did look for a parking sign, saw one above my car as indicated in the photo and did not consider that I might have to hunt down another parking sign for corroboration.

I would have continued contesting the ticket except for the fact that we donated our car to a charity a few weeks ago and didn’t want to cause them any trouble during their attempt to sell it. Perhaps the City of Sacramento will use the revenue from my citation to put up the proper signage…

Wikipedia used JavaScript and CSS to overlay their message on every page, so users saw the content of the page for a second or so and then it was completely obscured. BoingBoing returned a HTTP 503 “service unavailable” for every URL on their domain. Mozilla returned a 503 for their main URL. Imgur and Craigslist simply changed the content on their main pages and continued to return HTTP 200s. reddit redirected every URL on their domain to their protest message via a HTTP 302 “temporary redirect”. So which, if any of these is “correct”?

In this case, the HTTP 503 is the best option because it conveys two important bits of information:

1. The server is unable to handle the request
2. The situation is temporary

The HTTP 302 is dangerous because it conveys two bits of information, one which is correct and the other which is a lie:

1. The requested resource was found, but under a different URI
2. The situation is temporary

The difference likely doesn’t matter much to a person using a web browser, as in the worst case it might cache the 302 response. If the redirect changes on the server a user re-requests the URI, the cached response might be returned. At this point if the user thinks the page isn’t valid e.g. seeing a January 18th SOPA protest page on January 19th, it’s easy enough for the user to reload the page and retrieve the new redirect target.

However, a web crawlers like Googlebot, DuckDuckGo Bot etc. will store the contents of the page for consideration in their respective search index and may not visit a given URI very often. So if one were to configure their site to issue a 302 or 200 with alternative content during the time a web crawler visited the URI, it could render one’s page virtually inaccessible via a search as the bot would think the temporary content fetched was valid and not retry again until it is scheduled to visit the URI again. In contrast, returning 503 will likely cause the bot to break out of it’s normal scheduling cycle for a given URL such that it retries it sooner rather than later.

So next time you decide to stage a blackout, remember there are clients other than humans with web browsers accessing your site!

It’s similar in style to the Khan Academy, and it definitely took my understanding of Bayes’ Rule, Hidden Markov Models, etc. to another level. Sure there are tons of books and tutorials out there, but I feel there’s a limit to what one can learn without practice. One form of practice is to create a toy programming project, the main downside of which is never knowing if you’re doing it “right”. Contests ala Netflix Prize are another which at least provides better/worse feedback. The main benefit of online courses though is one can practice solving problems and know whether or not an answer is correct.

I didn’t participate in any of the discussions, though I did find the group website useful for clarification on a few of the problem statements. Even so, there were still a couple of questions throughout the homework and exams which I misinterpreted, leading to an incorrect answer; however, the largest cause of incorrect answers seemed to be my inability to punch numbers into a calculator correctly…I guess I haven’t gotten any better at that since I was last in school :S

An unexpected realization was how much I value flexibility in scheduling. There were definitely a few weeks I had to scramble in order to make it through the videos and homework before the deadline. Funny how the addition of one more deadline per week caused me so much grief. I was interested in the material, but some of it took quite a bit of time to get through (especially once I slowed down after rushing through a couple of homework assignments and doing poorly).

While one could certainly write programs to solve some of the quiz/homework/exam problems, all of them were solvable with pen and paper. The lack of problems requiring writing code was one of the top complaints. They held an AI competition; I didn’t participate as I feared that would take up too much of my time. However, towards the end of the class, the instructors provided an optional quiz which required writing some code: Optional Natural Language Processing Problems. This seemed like a much less time consuming set of questions and I would also get instant feedback.

The first problem was easily solved with brute force. My basic idea was to try every possible rotation and see which one yielded the most words which appear in an English dictionary.

#!/usr/bin/python

import sys
import string

# Problem text, all lowercase, no punctuation.
SENTENCE = 'esp qtcde nzyqpcpynp zy esp ezatn zq lcetqtntlw tyepwwtrpynp hld spwo le olcexzfes nzwwprp ty estd jplc'

# Parse the system's English dictionary into a set.
def WordsToSet():
  word_set = set()
  for w in open('/usr/share/dict/words'):
    word_set.add(w.strip().lower())
  return word_set

def p1(word_set):
  # Setup character position lookup table.
  chars = string.ascii_lowercase
  num_chars = len(chars)
  letter_positions = dict([(c, i) for i, c in enumerate(chars)])

  # Track the rotation that yields the most real words.
  max_word_count = 0
  best_i = 1
  sentences = {}

  # Try each rotation.
  for i in xrange(1, 25):
    temp_word_count = 0
    decoded = []

    # Build up a decoded sentence.
    for char in SENTENCE:
      if char == ' ':
        decoded.append(char)
      else:
        decoded.append(chars[(letter_positions[char.lower()] - i) % num_chars])
    sentences[i] = ''.join(decoded)

    # See how many words in the sentence exist in the word set.
    for word in sentences[i].split(' '):
      if word in word_set:
        temp_word_count += 1
    if temp_word_count > max_word_count:
      best_i = i
      max_word_count = temp_word_count

  # Output our best guess.
  print best_i, sentences[best_i]

def main(argv):
  word_set = WordsToSet()
  p1(word_set)

if __name__ == '__main__':
  main(sys.argv)

The second problem proved to be a bit more difficult. It couldn’t be brute forced since there were 18 “strips” of characters and thus 18! possible combinations (6,402,373,705,728,000 so if one could generate 1 every millisecond, it’d take 200,000+ years and that doesn’t even factor in the evaluation of whether the combination is gibberish). To make life easier, I first got rid of all the punctuation, lowercased all the characters, and converted the “strips” of characters into a Python list of lists. My idea was to pick the first strip, try the remaining 17 to see which one “fit” best before or after strip 0, then try the remaining 16 on the joined strips, etc. For best “fit” I would try to maximize the number of legitimate words which could be composed by the left or right most series of characters (whitespace delimited).

Initially it seemed like counting a partial word match with fewer than 3 characters wouldn’t be useful e.g. “er” appears in tens of thousands of words. That wasn’t good enough to keep my program on track though, as it would get a few strips aligned correcting and then veer off track creating sequences of characters that were found in many words, but ended up being gibberish in this example. So I had to augment the solution to detect when a sequence of characters should appear at the beginning of a word (whitespace before the sequence, but not after) or when a sequence of characters should be matched exactly (whitespace before and after the sequence). In this case, I didn’t have to detect whether a sequence of characters should appear at the end of a word (whitespace after the sequence, but not before), but it would be trivial to add.

Another necessary feature of my model was the ability to short-circuit if more than 1 row of characters would result in zero possible words. At first I had the threshold set at more than 0, but “shannon”, a last name as it turns out, is not in the dictionary. The code below takes about 90 seconds to solve the problem and could easily be optimized to reduce the number of loops or lookups in the dictionary, but I think those changes obscure the heart of the algorithm. Below is the code as well as a copy of the output generated by the program as it solved the problem.

#!/usr/bin/python

import sys
import string

STRIPS = [
  ['de',' ',' f','cl','nf','ed','au',' i','ti',' ','ma','ha','or','nn','ou',' s','on','nd','on'],
  ['ry',' ','is','th','is',' b','eo','as',' ',' ','f ','wh',' o','ic',' t',' ',' ','he','h '],
  ['ab',' ','la','pr','od','ge','ob',' m','an',' ','s ','is','el','ti','ng','il','d ','ua','c '],
  ['he',' ','ea','of','ho',' m',' t','et','ha',' ',' t','od','ds','e ','ki',' c','t ','ng','br'],
  ['wo','m ','to','yo','hi','ve','u ',' t','ob',' ','pr','d ','s ','us',' s','ul','le','ol','e '],
  [' t','ca',' t','wi',' m','d ','th',' a','ma','l ','he',' p','at','ap','it','he','ti','le','er'],
  ['ry','d ','un','th',' ','io','eo','n ','is',' ','bl','f ','pu','co','ic',' o','he','at','mm'],
  ['hi',' ',' ','in',' ',' ',' t',' ',' ',' ',' ','ye',' ','ar',' ','s ',' ',' ',' ']
]

def WordsToSet():
  word_set = set()
  for w in open('/usr/share/dict/words'):
    word_set.add(w.strip().lower())
  return word_set

# Determine whether word is, starts with or contins prospective_word.
def WordMatch(word, prospective_word, exact, startswith):
  if exact and prospective_word == word:
    return 1
  elif startswith and word.startswith(prospective_word):
    return 1
  elif not exact and not startswith and prospective_word in word:
    return 1
  return 0

def ProspectiveWord(i, j, row, prefix):
  # Construct a new result based on the current result row, the ith row and,
  # jth column of STRIPS and whether the new set of characters should be
  # positioned before or after the current result row.
  if prefix:
    prospective_result = STRIPS[i][j] + row
  else:
    prospective_result = row + STRIPS[i][j]

  words = prospective_result.split(' ')
  prospective_word = ''
  exact = False
  startswith = False
  if prefix:
    if words[0] == '':  # There was a leading space.
      prospective_word = words[1]
      if len(words) > 2:  # And a space after prospective_word.
        exact = True
      else:
        startswith = True
    else:
      prospective_word = words[0]
  else:
    if words[-1] == '':  # There was a trailing space.
      prospective_word = words[-2]
      if len(words) > 2:  # And a space before prospective_word.
        exact = True
    else:
      prospective_word = words[-1]
      if len(words) > 1:
        startswith = True
  return prospective_word, exact, startswith

def p2(word_set):
  # Set the first column of STRIPS as the current result.
  result = [row[0] for row in STRIPS]
  # The order of j's, not used to solve the problem, but useful to print at the
  # end. i.e. the order of the STRIPS.
  strip_order = [0]
  # Keep track of the unused j's i.e. columns of STRIPS
  remaining_strips = set([j for j in xrange(1, len(STRIPS[0]))])

  while len(remaining_strips) > 0:
    # Make one pass placing column j of STRIPS before the current result and
    # then another pass placing it after.
    for prefix in [True, False]:
      # Keep track of the best j and highest possible probablistic word count.
      best_j = 0
      best_word_count = 0
      for j in remaining_strips:  # Columns
        word_count = 0
        # Keep track of the number of rows without probable words. Rows which are
        # not checked i.e. without a prospective word of length > 2 or where
        # STRIPS[i][j] is a space (doesn't add any new information) do not count.
        rows_without_words = 0
        for i, row in enumerate(result):  # Rows
          # More than one row without probable words means this column is likely
          # in the wrong position.
          if rows_without_words > 1:
            break
          if STRIPS[i][j] == ' ':  # No new information, continue to next row.
            continue
          prospective_word, exact, startswith = ProspectiveWord(i, j, row, prefix)
          # Too short a word/word fragment, continue to next row.
          if len(prospective_word) < 3:
            continue

          # Total up all the prospective word matches and update the state.
          temp_word_count = 0
          for word in word_set:
            temp_word_count += WordMatch(word, prospective_word, exact, startswith)

          if temp_word_count == 0:
            rows_without_words += 1
          else:
            word_count += temp_word_count

        # More than one row without probable words means this column is likely
        # in the wrong position.
        if rows_without_words > 1:
          continue

        if word_count > best_word_count:
          best_word_count = word_count
          best_j = j

      if best_j != 0:
        if prefix:
          strip_order.insert(0, best_j)
        else:
          strip_order.append(best_j)
        for i in xrange(len(result)):
          if prefix:
            result[i] = STRIPS[i][best_j] + result[i]
          else:
            result[i] = result[i] + STRIPS[i][best_j]
        remaining_strips.remove(best_j)
        print 'strip_order: ', strip_order
        for row in result:
          print row
          print ' '

def main(argv):
  word_set = WordsToSet()
  p2(word_set)

if __name__ == '__main__':
  main(sys.argv)

Here’s what the output looks like so you can see it building up the solution:

strip_order: [6, 0]
aude
eory
obab
the
u wo
th t
eory
thi

strip_order: [6, 0, 15]
aude s
eory
obabil
the c
u woul
th the
eory o
this

strip_order: [3, 6, 0, 15]
claude s
theory
probabil
of the c
you woul
with the
theory o
in this

strip_order: [3, 6, 0, 15, 11]
claude sha
theory wh
probabilis
of the cod
you would
with the p
theory of
in this ye

strip_order: [3, 6, 0, 15, 11, 13]
claude shann
theory whic
probabilisti
of the code
you would us
with the pap
theory of co
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18]
claude shannon
theory which
probabilistic
of the code br
you would use
with the paper
theory of comm
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2]
claude shannon f
theory which is
probabilistic la
of the code brea
you would use to
with the paper t
theory of commun
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14]
claude shannon fou
theory which is t
probabilistic lang
of the code breaki
you would use to s
with the paper tit
theory of communic
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17]
claude shannon found
theory which is the
probabilistic langua
of the code breaking
you would use to sol
with the paper title
theory of communicat
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5]
claude shannon founded
theory which is the b
probabilistic language
of the code breaking m
you would use to solve
with the paper titled
theory of communicatio
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7]
claude shannon founded i
theory which is the bas
probabilistic language m
of the code breaking met
you would use to solve t
with the paper titled a
theory of communication
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4]
claude shannon founded inf
theory which is the basis
probabilistic language mod
of the code breaking metho
you would use to solve thi
with the paper titled a m
theory of communication
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12]
claude shannon founded infor
theory which is the basis o
probabilistic language model
of the code breaking methods
you would use to solve this
with the paper titled a mat
theory of communication pu
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12, 10]
claude shannon founded informa
theory which is the basis of
probabilistic language models
of the code breaking methods t
you would use to solve this pr
with the paper titled a mathe
theory of communication publ
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12, 10, 8]
claude shannon founded informati
theory which is the basis of
probabilistic language models an
of the code breaking methods tha
you would use to solve this prob
with the paper titled a mathema
theory of communication publis
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12, 10, 8, 16]

theory which is the basis of
probabilistic language models and
of the code breaking methods that
you would use to solve this proble
with the paper titled a mathemati
theory of communication publishe
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12, 10, 8, 16, 1]
claude shannon founded information
theory which is the basis of
probabilistic language models and
of the code breaking methods that
you would use to solve this problem
with the paper titled a mathematica
theory of communication published
in this year

strip_order: [3, 6, 0, 15, 11, 13, 18, 2, 14, 17, 5, 7, 4, 12, 10, 8, 16, 1, 9]
claude shannon founded information
theory which is the basis of
probabilistic language models and
of the code breaking methods that
you would use to solve this problem
with the paper titled a mathematical
theory of communication published
in this year

6:23p bee musketeer
6:56p butterfly and skeleton
7:11p airplane
7:30p spider girl?
7:35p fireman
7:40p 2 cowboys
7:41p 3 teenagers, no costumes
8:42p 3 teenagers (different ones), no costumes

I left our lit Halloween decorations along the 8ft path to our front door out until 10p and our porch light on until 11p. One can walk to 2 elementary, 1 middle and 1 high school within 0.7mi of my house; I don’t live on a busy street and there is a sidewalk. Many of the kids don’t even like to go to the mega-trick-or-treat zones; my wife works at an afterschool program and they complain it’s too crowded and easy to get separated from their friends; however, the parents feel it’s “the thing to do”. Our neighbors also seem to think the number of trick-or-treaters dropped from last year and I’m sure they have a bunch of excess candy just as we do. There’s a clear market inefficiency.For the first month+ of the Occupy Wall Street protests, I managed to turn a blind eye, blissfully occupying myself with baseball pennant chases, painting close to 1,000 sqft of outdoor fencing, Stanford’s Online Class – Introduction to Artificial Intelligence, etc. One evening, while walking back from the grocery store I expressed interest in seeing what one of these Occupy camps looked like and my wife mentioned if we walked down Martin Luther King Jr. Way (in Berkeley, CA) we’d pass by Occupy Berkeley. There wasn’t much activity, granted it was a Thursday night, just a handful of tents and some signs.

The next day I took a stroll by Occupy San Francisco along the Embarcadero, and it looked as if a number of homeless people decided to hang out together. I have nothing against homeless people, but I don’t think the many of the Occupy San Francisco population were ‘protesters’. Perhaps homelessness is a by-product of the state of affaires the Occupy Wall Street movement wants addressed; but I imagine most of the people I saw at Occupy San Francisco were homeless first and protestors second, if at all. I passed on visiting the Occupy Oakland camp due to the violence. Meanwhile the topic of “the other 99%” kept coming up more and more in conversation, such that I could no longer ignore it.

Based on the IRS Adjusted Gross Income Statistics my yearly income has put me in the top quintile (20%) ever since I graduated from college. This is not a result of me being especially brilliant or even lucky as if I were either of those I would probably have dropped out of college and joined the 1% during the dot-com bubble instead of catching the crash at the end. The two dominant factors are likely that I live in one of the most expensive areas of the country (San Francisco Bay Area) and work in an industry (computer software) which has higher than average salaries. In the last few years I’ve probably moved up into the top 10% and routinely interact with people who are currently in the top 1%, or have been in the past (you can check yourself for the current year via the Wall Street Journal’s “what percent are you?” calculator).

So in a sense, the current system works for me i.e. I can work within its constraints to do most of what I want to do. Note I do not aspire to have a Gulfstream 500 (I haven’t even ever flown first-class), houses in 5 different countries, or even retire early at this point. However, I realize the system has also failed many people, myself included to a degree as I am reluctantly a landlord due the the mortgage lending situation swinging wildly from giving loans to anyone who can breath to only giving loans to people with nearly perfect credit and a whole bunch of assets. Of course it could be much worse, I could not have a place to live or the ability to feed myself, but at the same time I’m contributing (proportionally) a great deal financially back into the system via taxes, charitable donations, etc. And I’m willing to contribute more because I know that not all work is valued appropriately and infrastructure is important to everyone’s well-being whether they use it directly or not.

It frustrates me the more I think about the Occupy* movement because while I agree with their general sentiment that what has happened with the economy recently sucks and should not have happened, I fail to see how they’re changing anything for the better. I suppose the current system of mass congregation to Halloween trick-or-treat zones works for the people on those streets who enjoy handing out disproportionate amounts of candy, but it doesn’t work for me. Maybe I should have tried occupying some of their front porches trying highlight the inequalities.

Instead, next year I will attempt to increase the demand in my area. I’m thinking about either stepping up to handing out full size (or perhaps king size?) candy bars and/or fancier Ghirardelli chocolate; perhaps word will spread that our street is worth visiting. Another thought I had was to put up neon orange signs, much like is done for a garage sale, advertising the bounty that awaits those willing to broaden their horizons. Perhaps our neighbors would be willing to join us in increasing the Halloween value of our block.

As silly as it sounds within the context of handing out Halloween candy, the general ideas i.e. increasing one’s value to society, making people aware of it and attempting to get others to join your cause seem like the only legitimate ways to change the situation. I wish the Occupy* movement would focus their efforts on something more productive than civil disobedience.

The male tarantulas come out around dusk towards the end of September and throughout October. We rolled up to the Mitchell Canyon entrance in Mt. Diablo State Park a bit before 6p and headed up the trail towards Deer Flat. After about 10 minutes, we came across our first tarantula, getting harassed by two families. Tarantulas in general, and especially the species found on Mt. Diablo, Aphonopelma smithi, are not very dangerous or aggressive. Because of this, many people attempt to handle the spiders. The recommended method is to place your hand on the ground in the spider’s path and allow it to walk over it. Raising it off the ground or attempting to touch it directly will likely scare the tarantula causing it to fling its urticating hairs and/or bite. Neither this group, or the next one we saw 10 minutes later seemed to heed this recommendation.

It’s actually quite tiring to continuously scan the trail trying to spot a brown tarantula at dusk on the brown dirt trail, amid the dirt covered rocks and poop. At 6:30p we turned around to head back, the possibility of seeing a spider outside of a tarantula jam (see bear jam) fading fast with the declining light. About 5 minutes later we happened upon a tarantula resting on the side of the trail. After snapping a few photos we hustled back to the car.

Shot at f2.0 due to the very low light conditions.

Dinosaur for scale.

If you decide to go, there is a $6 entrance fee and even though the sign says the park closes at sunset, you don’t have to worry about getting locked in the parking lot (the entrance gate is locked but the exit has no gate, just the “don’t reverse or your tires will pop spikes”). You shouldn’t have to walk far, we probably only walked a bit more than 1 mi up the trail. On the off chance you fail to spot any spiders, you will still be treated to some nice fall foliage.

• 3 Points: For every $1 spent on Amazon.com
• 2 Points: For every eligible $1 spent at gas stations, restaurants, drug stores, and office supply stores
• 1 Point: For every other $1 spent

On the plus side the card doesn’t charge an annual fee; on the minus side, the rewards certainly do not rank amongst the best in terms of points conversion i.e. most rewards convert at a value of $1 per 100 points e.g. $50 statement credit is 5,000 points. The only ones which convert at a higher rate are airline tickets e.g. a roundtrip continental U.S. flight up to $400 in value is 25,000 points or a roundtrip flight anywhere up to $1,600 is 90,000 points. It’s lame that some of the rewards are gift cards to restaurants or gas stations since it’d be better to charge those purchases on the card, get 2 points per $1 and then convert them into a statement credit. Even worse are the Amazon.com gift cards as money spent on Amazon.com purchases converts at a 3-to-1 ratio.

Lately Amazon has been pushing “pay with points” when during checkout which allows one to convert points at the 100 to $1 rate to pay for purchases. Anyone who does this loses money just as they would with the restaurant and gas station gift card redemption. The worst part is Amazon makes it so easy to lose money as there is no waiting for a gift card to arrive in the mail. If it isn’t clear why this is such a bad deal, here’s an example.

Let’s say someone spends $5,000 each year on an Amazon.com Visa card, $2,000 of which was on Amazon.com, $1,000 on gas and restaurants and $2,000 on other purchases. At the end of the first year that person would have (2,000 x 3) + (1,000 x 2) + 2,000 = 10,000 points. That could be redeemed for a $100 statement credit or $100 restaurant gift card or spent on $100 worth of Amazon.com purchases.

If the person pays with points on Amazon.com, then the next year they have still acquired $5,000 worth of products but only have (1,900 x 3) + (1,000 x 2) + 2,000 = 9,700 points, good for $97 worth of rewards. Granted that’s only $3 but why give away $3 or reframed, 3% of one’s rewards for no reason? Even more importantly, why would a company that in so many other ways focuses on what’s best for the consumer, provide an option that clearly doesn’t benefit the consumer?

• My most beautiful post
• My most popular post
• My most controversial post
• My most helpful post
• A post whose success surprised me
• A post I feel didn’t get the attention it deserved
• The post that I am most proud of

Even though Tripbase is focused on travel, I won’t necessarily restrict my selections to travel related posts.

My most beautiful post
I think that would have to be the Final Launch of Space Shuttle Atlantis (well as it turned out, the penultimate launch). Not only are the photos beautiful, but the opportunity to experience a launch of a space shuttle has sadly gone the way of the Dodo.

My most popular post
According to Google Analytics it’s Predicting When Derek Jeter’s 3000th Hit Will Occur. I was completely wrong, mainly due to Jeter going on the disabled list and missing 20 or so games.

My most controversial post
Definitely IBM Watson Vs. Google Search. In addition to the comments on my blog, “kenjackson” slighted me on Hacker News. What cracks me up the most is how people who don’t work at Google think they authoritatively know how Google works and what it does and doesn’t do with regard to A.I. and natural language processing.

My most helpful post
Most likely GPX Elevation Profile Plotting With The Google Chart API. Based on the traffic and when my page shows up in search results I think most people find the GPX parsing/elevation calculation code useful as when I wrote it there weren’t many examples that just worked e.g. people would describe calculations or enumerate the mathematical formulas, but didn’t commit them to code.

A post whose success surprised me
I don’t really know if any of my posts can be considered a “success” but maybe my research into some Art History after a visit to the musée du Louvre fits the bill.

A post I feel didn’t get the attention it deserved
After re-reading my waxing philosophical post about Redwood Trees surprised me. According to Google Analytics not many people have read this post, but I think it’s probably one of my best.

The post that I am most proud of
Idea-wise, the previous post may be the one I’m most proud of; however, in terms of travel and personal achievements I’d have to say my Great Expectations post takes the cake as I wove together 3 mentally and physically challenging endeavors and question why I do the things I do.

With rough target dimensions of 18″ high, 12″ wide and 6′ in length, I went to our local Home Depot to check out the available lumber and cost. One of the many things in my life classifiable as “obvious but overlooked” is wood density varies by species. This became readily apparent after picking up a piece of Poplar, deciding it didn’t look nice and then picking up the same size piece of Oak which weighed twice as much.

After walking around and around searching for light wood that looked nice and would fit in the car without having to run down an employee to cut it down to size, I ended up with Redwood. An additional upside with Redwood is its resistance to insects, fungi and fire. A downside is its additional cost. I mitigated that by using some rough cut fence pickets in on the underside of the seating area which were 1/4 the price of the nicely finished planks.

Here’s a photo of the raw wood, 4 finished 1×6 boards, 3 1×6 fence pickets, 1 4×4 post and 3 2×2 posts (I actually ended up not using one of the 2×2 posts).

And here is a photo of all the pieces of wood cut down to size.

Here are the assembled “legs”.

This is a shot of the legs and seating area cross supports getting attached to the fence pickets.

And here are a few shots of the finished product.

If I had to do it over I probably would have spent more for 2x6s instead of layered 1x6s for the seating area mainly for aesthetic reasons i.e. I wouldn’t have had to use so many visible screws. Other than that, the bench has performed well, having been used 3 times with no casualties.