The server got hacked over the weekend because I had an old version of Drupal on an old version of Ubuntu (yes, Sitepoet is coming along nicely and should fix that…). So in the rebuild I made two changes to the system to help out when things like this happen again.

First, there is now an el cheapo Rackspace cloud instance functioning as a MySQL replication slave so I have a live backup of the database. This is because after the hack was discovered by the fine folks at Quantact they rate-limited the VPS so the attacker would see less use of the instance and not be able to flood anyone (a good practice). However, what that also did was cause mysqldump to fail to get a backup of my database (about 1.1GB). I had a system doing a nightly DB dump and rsync of important files, but that dump was a day old, not live. Now it’s live.

Second, on the advice of a friend I looked into Drupal’s memcache module and a local copy of memcached. It’s a rather nice little module that simply moves all cache get/set calls away from using the database and into using the memcached server instead. Sure, it takes up 64MB of RAM on my wee 512MB instance, but, well, look at the numbers:

Gets 2.20/s; 121,255 gets (82.21%); 121,255 hits (100.00%) 40,059 misses (33.04%)

That’s an 82% hit ratio for page content. That’s crazy. The site is amazingly faster now that it’s not I/O-bound under load and should it ever get hammered it will cache the queries for the hammered page in memory and not get bogged down in the database. Fantastic. Now I just need content worthy of getting hammered…

PS: Oh, if you run multiple sites off the same install (as I do) you’ll want to put a prefix on the cache keys lest you start serving out cached data from one site to another, such as the compiled page template…

Remember, Mac Heads, iChat lets you both share and control the screen of someone you’re talking with. So if you have electronically-inept family and friends just get ‘em to fire up iChat, request to share the screen, and fix the problem. No Mobile Me, no VNC, no Apple Remote Desktop, etc. All built-in and easily-used.

Yes, this means I have to close Adium, but I’ll pay the price…

Every time the AT&T tethering tax comes up, someone whines “it should be included!” Well, yes, it should. But it’s not. Does that instantly mean it’s a bad deal? Not really. Don’t look at the cheapest possible interpretation. Instead, look at what it would cost to get the feature any other way (which is, of course, how they’re pricing it to get the most out of you).

Let’s do some numbers.

Yes, you can whine that it should be included, and you’d have a point. But it’s still cheaper for remote access than the alternative. And with 4.3 coming out with the hotspot, you get another freebie if you have a 3G iPad…

You save five bucks and get a hotspot for your computer to boot. Bada-boom. And if you’re still on the unlimited plans (and aren’t using them past 2GB/month) then you save $5-10 more with the switch!

So no, not the best deal, but there ARE ways to make it work for you if you think about it. I did the math and, for me, it works out great when the hotspot feature comes out. For $45 I get access for all my devices for light use anywhere in town. Email, web, some video and files, but not a lot. The bump to 4GB of data? Icing on the cake. I was already sold.

It’s all about use cases, folks.

¹ This is the tier that includes 3G service.

While I take a short hiatus from iPhone programming, I’m finding that I’m now aware of some parts of the mobile community that I wasn’t before with my head down and chugging along in Xcode. Namely, this nearly-blind worship of Android from the FOSS community, in spite of most any valid criticisms. In fact, it appears to rival the blind fanaticism of the Apple community in some ways, and in some ways is a direct result of the Apple community and company.

Apple fans are known for their reality-distancing opinions and astounding ability to completely detach themselves from truth and fact as necessary, much like fundamentalist Christians when presented with old passages about mixed textiles and stoning women. So it should come as no surprise that there are a ton of iPhone and iPad fans out there that are head-over-heels in love with the damned things and can’t comprehend why any of the AppStore’s numerous misguided policies are anything but “right for the platform” or some other rosy-goggled bullshit line. However, with every fanatic on a given side there will be an equal and opposite fanatic against it. This was true with the old Mac vs. Windows battles and it’s true today with iPhone. The problem is that before Android came to market, there wasn’t another side to back (Windows Mobile? Please. WebOS? Ha!).

So, enter Android, made by the third giant. What amazes me about this is the brilliant move that Google made in making the operating system open source from the start. This not only made it amazingly easy for phone developers to get in and build devices with it, but it also endeared the FOSS crowd to Google instantly (again). Easy adoption, rabid supporters, high availability — a clear winning combination. Indeed, adoption is really taking off and it’s becoming the default OS for new smart phones that would rather not pay licensing fees or have to deal with writing their own shitty OS to compete with the iPhone.

A large portion of the time spent uploading a large directory with utilities like scp and sftp (and sometimes rsync) is sending all the meta information over for the various files (and creating them). For some reason, this sometimes takes a purely silly amount of time.

The upside is there’s a simple trick that will let you upload things very quickly. The tar command is optimized for dealing with lots of little files and does it far, far better than the other commands. It’s not network enabled, but ssh does understand pipes…

tar cz MyDirectoryOfLittleFiles | ssh remote.host "tar xz"

Ta da! Full-speed uploads with tar on either end doing all the file work and ssh just encrypting the stream. For me, over local connections, this gives a much more even upload experience than scp alone ever did, especially for large directories of small files (source, for example).

Downside: it doesn’t do syncing like rsync. If this fails in the middle, you get to start over or switch to rsync to finish it.

So I’m reading in a CSV file on the iPhone. I figure it’ll be good to link up the column title with the values by returning an array of dictionaries, each keyed to the column title. Then, I’m going to save that data into a Core Data SQL store so I can get it out of memory. Fantastic, yes?

Should have worked. Apple broke it.

Two things broke it, actually. While the strings that make up the column names are immutable strings that could have been simply retained as the dictionary key, Apple chose to copy them instead. When you have over a thousand records, that adds up fast. On the phone, it adds up faster. In fact, it adds up to about 1MB of memory wasted copying objects that cannot, by their nature, change.

The second failure happens during Core Data’s save phase. The SQL store now proceeds to copy all the values in each entity before saving them, keeping them all around in an autorelease pool that is flushed after save. That adds another 2MB to the memory usage right there.

So we go from a 500KB app heap at startup to 1MB after loading the string that represents the CSV file to 2.3MB after creating the initial array of data and then to 3.2MB of data after transitioning to the dictionary (same data, all unchanged and immutable objects). Then to top it off, Core Data brings the usage up to 4.5MB just to save everything. When it’s all said and done the heap goes down to 1MB.

So, yes, now I get to go through and create autorelease pools and flush them to capture objects behind-the-scenes and then ensure all my objects are flushed before telling Core Data to save, but none of this would be needed if Apple were simply smarter about the fact that their frameworks would be used in a low-memory environment that prefers to pass around immutable objects rather than making whole copies of them.

Copy bad. Retain good.