May 31 2008
∞
parallelize: Shell utility to execute command batches in parallel
Warning to non-geek readers: You probably want to skip this post.
I have an 8-core Mac Pro, and I sometimes have a big list of CPU-intensive commands I want to run (image resizing with ImageMagick, video/audio encoding with ffmpeg, etc.), but they’re usually single-threaded or just very bad at using multiple CPU cores.
I assumed that there must be some clever little shell utility to run 8 of these commands at once from a big list until the batch is done, effectively utilizing my 8 cores.
I couldn’t find such a utility, so I wrote it myself. And I’m sharing it with you, because there’s a chance you’ll find it useful, too. (Even if it already exists, at least I got to learn how to use the pthread library and flex my atrophied C muscles.)
Here’s the C source code and documentation. It’s only 57 lines of code (preceded by 22 lines of documentation).
Works at least on OS X 10.5, CentOS, and Debian. Probably works with any Linux or BSD system. (In theory, it should work with anything with pthreads.)
Usage:
parallelize {thread-count} < big_list_of_commands.txt
Demo:
(for i in 1 2 3 4 5 6 7 8 9 10; do echo "echo 'Delay $i'; sleep $i ; \
echo 'Done $i'" ; done) | parallelize 4
Let me know if you find bugs (especially parallelism bug potential), or if you just find it useful and want to say hi.
+
∞
And David comes out of nowhere with this awesome photo!
You should get fevers more often.
+
∞
“ The unspoken truth is that these businesses don’t hire illegal aliens because they can’t find American workers, they hire illegal aliens because they don’t want American workers. And it has nothing to do with wages. Illegal aliens mean no workers’ comp claims, no age, race or sex discrimination lawsuits, no healthcare premiums, no unions, and no demands for raises, vacations or bigger offices. In fact, illegal immigrants are the perfect employees because they’re not employees at all; they’re corporate slaves.
— Slavery alive and well in U.S. (via azspot)
+
∞
7A with Tiff, Lee, and cbeth.
That’s a red-pepper eggs benedict with coffee and a mimosa for $12.50. Gotta love their weekend brunch specials. (brunch is Sat-Sun, 10am-5pm, and 7A is on the corner of 7th… and A)
+
∞
Daring Fireball: Mobile Me?
Looks like Apple is preparing to rename .Mac to “Mobile Me”.
.Mac is a decent idea that falls way short of its potential. I hope that a significant upgrade is in the works.
Synchronizing data between multiple Macs (home and work, desktop and laptop, etc.) is pretty painful outside of the realm of .Mac’s supported data types (contacts, calendar, mail). It would be amazing if, someday, there was a larger synchronization platform from Apple that would incorporate iTunes- and iPhoto-library bidirectional sync, plus maybe a general home-directory sync option (or just certain folders).
Imagine if .Mac/Mobile Me would store your libraries and data on the internet somewhere, handling backup and versioning automatically with a remote Time Machine interface, with seamless integration with iLife. I’d gladly pay $30-60/month for that.
+
∞
(via jacobbijani)
Hey, David’s name is on this box. I wonder what it is.
Wait, what? My name’s on this box, too! Am I speaking at something? (I truly have no idea what this is.)
(Update: Oh. Cool.)
+
∞
+
∞
“ It’s the 21st century, there’s not going to be a year in which it’s harder to copy than this year; there’s not going to be a day in which it’s harder to copy than this day; from now on. Right? If copying gets harder, it’s because of a nuclear holocaust. There’s nothing else that’s going to make copying harder from now on. And so, if your business model and your aesthetic effect in your literature and your work is intended not to be copied, you’re fundamentally not making art for the 21st century.
— Cory Doctorow (via poortaste)
May 30 2008
∞
Well, that ended tonight’s Rock Band party prematurely.
+
∞
Wilco: Kicking Television, Live in Chicago
This week’s experiment album.
I keep hearing about Wilco — now I’m finally trying them out. So far, so good. Very mellow, overall, especially after a week of Muse’s back catalog.