1. Make sure you have the appropriate codecs installed. I generally use the Perian codec package. I additionally find that some FLVs require QTPro to be installed; it's not very expensive.
2. Download the video somehow. Some sites, like Google Video, let you download a copy. Others, like YouTube, do not allow this. However, most embedded flash video can be grabbed via the technique in the bottom video in the demo videos at Perian.
3. Open the video in QuickTime. The video is now happily outside the browser.
4. Go to Window → Show A/V Controls; change the playback speed in the relevant window. I find that 2.0x generally works pretty well; the video will be faster and the audio is a little clipped but nicely de-chipmunked.
5. Enjoy your new lecture! The glacial discussion now arrives at a rapid-fire pace. You'll be too busy trying to keep up to play Desktop Tower Defense, and you'll be done in a half hour.

]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/439798812/overclocking-lecture.html http://joshua.schachter.org/2008/11/overclocking-lecture.html complaining Sun, 02 Nov 2008 00:14:21 -0800 http://joshua.schachter.org/2008/11/overclocking-lecture.html Ever since seeing a presentation by Dolores Labs about Amazon's Mechanical Turk, I've been itching for an excuse to play with the system.

I recently saw a thread that highlights the distinction between expected value and utility. Would you take a more likely but lower payoff instead of a less likely but higher payoff? Similarly, the St. Petersburg Paradox takes the problem to its logical extreme. By constructing a game that has a series of increasingly rare payoffs of increasingly larger size, a game with infinite expected value is created.

So I constructed 21 versions of the questions, varying the size of the dollars as well as the rate of payoff for the second outcome.

For one cent apiece, I sent the questions to be answered by one hundred people each, and collated the results. 2100 questions, three hours, and thirty dollars later, I have my results.

Clearly, people (or at least these Turks) begin to cross over at larger values, reaching equilibrium at around $1,000.

While this isn't the most groundbreaking work, it is nice to be able to generate an experiment and gather the results in the course of an evening and then have the results be so pleasing.

The Mechanical Turk is presented as a way to solve problems that are easily explained to people but difficult to implement for computers, frequently described as "artificial artificial intelligence." However, I think some of the most intriguing uses yet will be to explore the edges of our own uniquely human behavior and self-understanding.

A publish/subscribe architecture is natural to other problem domains such as instant messaging and financial data systems (Tibco, Reuters, and so on).

Similarly, Brad Fitzpatrick implemented something similar as a never-ending Atom feed a few years ago for Livejournal (sans XMPP, which wasn't as conceptually prevalent then.)

One important point in the presentation is that, for example, a single application would poll Flickr approximately three million times in a day to fetch only several thousand updates. At Delicious we saw a similar level of polling activity, made somewhat worse by speculative querying (hitting the URL information pages to see if there was any data for arbitrary URLs, which was generally unlikely.)

One solution that ocurred to me at the time was to build a simple callback system over HTTP. This would fall comfortably between full polling and full persistent publish/subscribe. The clever acronym even writes itself: PIMP Is Mostly Push, although maybe PRSS (Push RSS) would be slightly more polite.

Simply described, instead of polling frequently, a client would send a normal HTTP request with the resource to be subscribed to and an endpoint to deliver updates to: http://your.app/subscribe?resource=/some/user&callback=http://my.app/endpoint

Presumably the endpoint would then receive RSS item fragments when and only when that resource updated. For security, the exchange should include some kind of token, borowing from the appropriate protocols. The subscription would lapse after, say, 24 hours, or that could be passed in as a parameter.

In some ways this is slightly more elegant than the XMPP solution as neither side has to maintain a dedicated long-running process. A simple server-side implementation would justfetch items from a work queue and send out HTTP messages. A simple implementation on the client side would be a plain old web page that could accept and process a POST request. There are a number of people on inexpensive service providers who have at best web scripting hosting and not much else. The case where Delicious/Twitter/Flickr pushes my own items (and not much else) up to my blog is an important one. Additionally, there would not need to be any persistent TCP connections, which is probably more efficient in server resources (but less efficient in network resources; for billions of messages the TCP overhead becomes significant).

Of course, callbacks are totally infeasible for a variety of other uses, especially for mobile or desktop applications (which are likely to be firewalled). ]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/344892536/beyond-rest.html http://joshua.schachter.org/2008/07/beyond-rest.html ideas Thu, 24 Jul 2008 10:41:59 -0800 http://joshua.schachter.org/2008/07/beyond-rest.html Rhizome 2008 Benefit in New York City later this week. There are still tickets available, so you have no excuses for not attending.

]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/288942280/rhizome.html http://joshua.schachter.org/2008/05/rhizome.html Mon, 12 May 2008 12:14:15 -0800 http://joshua.schachter.org/2008/05/rhizome.html ]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/281727625/tag-mockery.html http://joshua.schachter.org/2008/05/tag-mockery.html complaining Thu, 01 May 2008 14:49:22 -0800 http://joshua.schachter.org/2008/05/tag-mockery.html So when I realized that tomorrow's imminent arrival of Stupid Internet Joke Day (was: April Fool's Day) would require the avoidance of all unnecessary internet contact, it also occurred to me that I may as well point out some common Funny Anti-Patterns. But that's been done to death - thank you, internet hipsters.

I merely wish to remind you all that elaborate hoaxes (press releases involving small company acquiring a large one, switching stylesheets with someone, etc) are immediately and transparently stupid. Instead, try to actually do something surprising. The ha you save might just be your own. ]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/261519870/stupid-internet-joke-day.html http://joshua.schachter.org/2008/03/stupid-internet-joke-day.html complaining Mon, 31 Mar 2008 12:57:34 -0800 http://joshua.schachter.org/2008/03/stupid-internet-joke-day.html However, a simple trick I learned early on is that even if you have only a single web server, a proxy in front can help out performance significantly. Through the simple expedient of buffering the communication with slow web clients, your potentially heavyweight (especially when mod_perl meant that each process was dozens or even a hundred megabytes apiece) and/or expensive Apache processes don't have to waste time serving every request for the entire length of time the client is connected. This allows you to run vastly fewer Apache processes.

In the past, I've used pound and perlbal. Pound is fast and lightweight, and allows routing based on the HTTP query; for example, everything under /img/ got routed to a high-speed thttpd instead of the Apache itself. Perlbal is much more configurable but slightly harder to get running, and the documentation was sparse.

These days, I'd also investigate nginx and varnish. Pen, a generalized TCP load-balancer with server affinity (connections will go to servers they've gone to recently in the past) is also quite interesting but will not help with the slow client problem. Finally, a second set of apache processes, configured to reverse-proxy via mod_proxy, will also do the trick. A ]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/221165432/proxy.html http://joshua.schachter.org/2008/01/proxy.html lessons learned Tue, 22 Jan 2008 00:16:32 -0800 http://joshua.schachter.org/2008/01/proxy.html fed up with PR folks spamming him.

Me too.

These were sent to the email address listed on my blog; I use tear-off addresses for subscribing myself to things.]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/177543339/unsubscribe.html http://joshua.schachter.org/2007/10/unsubscribe.html complaining Tue, 30 Oct 2007 20:54:13 -0800 http://joshua.schachter.org/2007/10/unsubscribe.html I find myself lately re-entering everyone I know into the system every year or two; I remember Six Degrees, Friendster, Linkedin, (I skipped MySpace -- I'm too old,) Facebook, Dopplr, Flickr, and so on. Brad Fitzpatrick seems to agree that this is an annoying waste of time, and says so in his thoughts on the social graph.

Most social systems never forget anyone. Given that recent behavior appears to send friend requests to anyone you've ever met even briefly, I find my contacts list ends up filled with people I don't really know. In many systems, removing someone from your list is either buried or simply impossible. Further, since these systems make implicit relationship information explicit, deleting someone becomes a loud signal. In real life you would merely back off a bit, but the systems only allow you to express a binary sort of relationship.

Therefore, switching networks becomes a way to regularly cleanse your contact list. There is evidence that younger internet users regularly start new instant messaging IDs; this likely serves a similar purpose.

So perhaps frequent switching is less a function of fashion but instead a coping mechanism to deal with the mismatch between reality and software.

]]> http://feeds.feedburner.com/~r/JoshuaSchachter/~3/139472801/elevator.html http://joshua.schachter.org/2007/07/elevator.html obsess Tue, 31 Jul 2007 22:38:33 -0800 http://joshua.schachter.org/2007/07/elevator.html When folks ask me where they need to go in NYC, I have a short list of places I send them to. I'm not saying these are the best or anything; I just miss them.

At Gray's Papaya on W 72nd and Broadway, the Recession Special, consisting of two hot dogs (just mustard) and a papaya drink, shared with my wife, is a reliable late-afternoon snack.

Pam's Real Thai is a tiny hole in the wall on West 49th Street and 9th Avenue; I always get the red curry with chicken. After eating chicken and associated vegetables, I feel so badly about abandoning the remaining sauce that I douse the remaining rice with it or just get a spoon and eat it like a sweet and spicy soup.

The omakase (chef's choice) at Sushi of Gari on Columbus at W 78th isn't deeply concerned with being authentic and I don't much mind; the sushi is amazing fish with all sorts of interesting garnishes. I my favorites are the salmon sushi with roasted tomato and the marinated tuna with pine nuts on a tiny crisp flake of fried nori. The Upper East Side location is supposedly better (Gari himself presides) but I've found the Upper West Side spot easier to get into on short notice.

I am told that the shakes at Shake Shack in Madison Square Park are amazing, but honestly. I always get the Shack Burgers, which are transcendently good. The only hard decision is whether to get the Single Shack, which has an excellent sauce-to-meat ratio, or the Double Shack, which has more of the tasty, tasty meat. The lunchtime lines are too long to deal with; go early (11:30 am or so) or late (2:30 pm) and the lines won't be too long.

Joe's Shanghai, on Pell Street, has amazing xiao long bao - tiny dumplings filled with a bit of meat and soup. They're a bit challenging to eat and there are a variety of strategies; I prefer to poke a hole in one and let it drain into a spoon, then add a bit of gingered vinegar, and then drink the soup and finally eat the remaining dumpling. There's an location on West 56th Street, which is much more expensive but equally good. Twice a year, a famous Taiwanese chef, whose xiao long bao are even better, makes an appearance at the Sheraton in Flushing, but the scheduling is just too difficult to work out.

When you walk down the hall and see someone you know, you raise your eyebrows to acknowledge their existence, and expect the same from them. If they don't reciprocate, you can plausibly tell yourself that perhaps they didn't see you, or were otherwise distracted. However, when you send someone an instant message, and they never reply, you can be reasonably sure they got it and are ignoring you. Thankfully, in the email world, we can at least blame the spam filter as to why you never replied.

It occurs to me that not every factoid gleaned from the constellation of behavioral data should be presented.

For example, the emminently social Twitter, happily informs me that while 34 people count themselves amongst my friends, only 31 of them care to be informed about I'm up to every day -- and then shows me who those folks are. While these lists are on different actual web pages, it's not a herculean task to figure out the actual people involved. Even though it's possible to show all the information, from a social perspective a degraded view would be better.

The first problem is that you will be tempted to use the internal identifiers in external URLs. I realize that RESTian canon indicates that every single object have its own identifier, and many new and whizzy frameworks generate simple create/lookup/update/delete user interfaces automatically.

URLs that include an identifier will let you down for three reasons.

The first is that given the URL for some object, you can figure out the URLs for objects that were created around it. This exposes the number of objects in your database to possible competitors or other people you might not want having this information (as famously demonstrated by the Allies guessing German tank production levels by looking at the serial numbers.)

Secondly, at some point some jerk will get the idea to write a shell script with a for-loop and try to fetch every single object from your system; this is definitely no fun.

Finally, in the case of users, it allows people to derive some sort of social hierarchy. Witness the frequent hijacking and/or hacking of high-prestige low-digit ICQ ids.

The second problem, in the case of MySQL, setting a column as auto_increment requires that there be a primary key placed on this column. It's not well explained in the documentation, but under InnoDB, the primary key is similar to a unique key, except that the rows in the database are stored in the sort order of the primary key -- this is why there may only be one such key. (Other database systems refer to this as a "clustered index"). This means if you are using it merely as a join identifier, but freqently do large queries based on some other column, the rows have to be fetched from all across the disk since they are not all together. As an example, in early implementations of del.icio.us, fetching all of the bookmarks for a given URL could cause tens of thousands of disk seeks even if there was an index on that column. As a datastore grows, the location of things on disk in relation to each other becomes an important consideration for scaling.