A visit to the Antipodes

I just returned from a business trip to Sydney, Australia, and Auckland, New Zealand. My second time in Australia, my first in New Zealand. I had a great time visiting with clients, colleagues, and seeing the sights. Below are some highlights of my trip.

Sydney

In Sydney, I stayed in a hotel right by Circular Quay, which made it ultra-convenient to hang out at The Rocks, get to the ferries, and get great views of the opera house and the harbor bridge every day. The weather was fantastic. Roughly 80 degrees Fahrenheit every day, blue sky, and light wind. Since I planned to be outside pretty much every chance I could get, I couldn’t have hoped for better weather.

The Rocks

No tourist visit is complete without spending some time at The Rocks, which is history, tourism, pubs, food, and shopping all in one place. I spent time at Phillip’s Foote (grill your own slab of meat), The Australian Hotel (wide beer selection, awesome pizza), and saw a bizarre-but-good U2 cover band at The Orient Hotel while enjoying a $7 Guinness, the cheapest beer I found on my trip. We ‘mericans are spoiled by good beer that is relatively cheap.

I also stumbled across some artwork that basically blamed the colonial revolt by the Americans in 1776 as the reason why Australia became a dumping ground for convicts in the early 19th century. England needed to find another home for them after their convenient offshore prison across the pond was no longer available. I had never made this historical connection before. Is it valid?

Hyde Park

A large park adjacent to Elizabeth Street, and right next to our office there. There was a large protest there the prior weekend, not a good time to be there for a tourist. Fortunately, the immediate turmoil had died down by the time I was there, and there was a neat photography art exhibit on display. Lots of families out enjoying a lovely spring equinox. As it should be.

Opera House

Everybody knows the opera house, and it’s quite impressive in person. It’s hard to avoid filling up your camera with photos of this thing from every angle. I did take the guided tour inside, which was interesting, but not necessarily awesome. Much of the history can be learned on your own, and probably the best way to experience the interior (which has several theaters / performance halls) is to just get tickets for a performance. There are over 1000 performances there each year!

Another view, close-up. The tiles are really interesting.

Harbor Bridge

I went for a jog each morning, and one of the days took me over the bridge to North Sydney and back. Pretty fun, in spite of the lungs full of diesel exhaust from the morning commuters. The bridge opened in 1932, and is absolutely massive. You can climb up the top of the bridge or walk up the stairs inside the southeast pylon. Both are commercial operations, and cost money. I didn’t do either. Simply being on the bridge a zillion meters from the water was cool enough for me.

Royal Botanical Gardens

The gardens are awesome. If you like a huge diversity of trees, plants, flowers, and birds … well, this place is amazing. A morning jog through this would be a great way to start every day, so you Sydneysiders are quite lucky to have this in your backyard. As for the bats? Well, I never found them. I guess the campaign to get rid of them is working.

NRL Preliminary Final

My colleagues and I were lucky to get tickets to the National Rugby League semifinal match between South Sydney and Canterbury at ANZ Stadium (the olympic stadium from 2000). Not knowing either club — or the sport itself, really — I bought myself a Rabbitohs hat and cheered for the underdogs. Who got crushed 32-8, after owning an 8-4 lead early and having all of the momentum.

I was one of about 12 Rabbitohs supporters in the crowd of 70000. That was kind of fun.

Oh Rabbitohs, you ruined my day!

My kids like the hat, though.

Bondi Beach

Love or hate the big city, having a beach like Bondi so close to town is pretty awesome. I spent a good part of the day there. The sand was a fine powder, great feel. Walked through the farmers market, bought a baked treat, enjoyed the heck out of it. Ate lunch with Scott (my colleague) at The Bucket List, outside and overlooking the water. Lovely way to spend a day.

Took a cab (spendy) there, walked to the train station on the way back and returned to the city via train. I think I liked the cab better.

Manly, and Sydney Harbor National Park

I went to Manly twice. Once, on a Friday evening, and had drinks at 4 Pines Brewing (awesome!) and pizza at Hugo’s (double-awesome). It was dark, and I didn’t get to see the town or the beach. So I returned on Sunday. Along with all of Sydney, I think. The place was crawling with people that day. But, as I carved my way through the crowds, I was able to see enough of the town to get a feel for the place. Lunch at BenBry Burgers took all of the pain of the crowds away.

I walked along the beach and found Shelly Beach tucked away from the hustle and bustle. That looked like a good place to spend a day. However, my goal was beyond Shelly Beach — Sydney Harbor National Park, which has a trail system that begins just above the beach. I walked all over these trails throughout the day, and made it to the headlands overlooking the cliffs before turning back and taking the trails back down the hill to the Manly town center.

Another trip to 4 Pines Brewing capped off the day. The special Märzen was a winner. It’s possible I may have ordered a second pint.

Manly is a 40-minute ferry ride from Circular Quay, or take a bus or cab. Take the ferry. Cheap and easy.

Oh, and I saw an echidna!

Auckland

In Auckland, my hotel was on the water — literally. Built on a pier jutting into the harbor. Oddly, also right next to the ferry terminal, as in Sydney. The weather was not quite so nice … maybe high 50s, overcast, and occasional drizzle. Just like home!

My schedule was much tighter in Auckland than in Sydney, so I saw less. But I still had a great time, and my colleagues there helped me make the most of my visit. A couple of evening jogs got me out on the harbor, which was very pretty.

I ate at Tyler Street Garage (hipster hangout, great pizza and beer), Degree (low key, diverse menu and beer), L’Assiette (only got as far as coffee and pastry), and Mexico (lamb & mint quesadillas … yum).

Piha

My colleague Tim took me to Piha, and we hiked to Kitekite Falls. Very cool. I really enjoy hiking, and this was a great little walk with a nice waterfall reward at the end. Learned about the native plants and trees, in particular the silver fern — which I recognized to be the symbol used by the All Blacks.

Rangitoto Island

A young volcano in the middle of the harbor? I’m there. My colleage Serge recommended this (as did others), and I enjoyed the heck out of this. My schedule was a bit tight, and I hoped to be able to hike to the top and back in the 3-hour window between ferry visits. The literature suggested this might not be possible, but I tried it anyway. I power-walked and ran up it in 45 minutes. 🙂 Left me plenty of time to take in the vista, and leisurely stroll down. Didn’t get a chance to visit the lava caves, which probably didn’t matter since I didn’t have a flashlight.

 

Wrap-up

Thanks to Sarge for the geographical reference.

Sadly, in Sydney, I was unable to locate 42 Wallaby Way, and no indication that P. Sherman lives or works there. I blame the iOS 6 maps app.

Time Machine sucks, use rsync instead

Mac OSX has a great built-in feature called Time Machine, which is designed to provide simple backup & restore functionality for your system. Time Machine does more than just keep a most-recent backup handy; it keeps track of changes to your files on a regular basis, and allows you to go back in time to a prior state of your filesystem to recover files that were lost — even those that were deleted intentionally.

Time Machine can write backups to an external drive (USB, FireWire, or Thunderbolt), or to a network device. Apple sells a network device called Time Capsule that works with Time Machine, but other vendors also provide NAS devices that integrate directly with the Time Machine software on the Mac client.

I’ve been Mac-only at home for six years, and have used Time Machine ever since it first appeared in Leopard, in 2007. Last year, I upgraded my backup strategy to point all machines to a Synology DS411j NAS. This has been an awesome little device, and I use it for more than just backup/restore.

Backups are pretty darn important

Your data is really important to you. Trust me. Maybe it’s your family photos or home movies. Or tax returns. Or homework. Whatever it is, it’s critical you keep at least two copies of it, in the event of accidental deletion, disk failure, theft, fire, or the zombie apocalypse.

Scott Hanselman has a nice writeup on implementing a workable backup and recovery strategy. I’ve done that, and you should do the same. If you use a Mac, Time Machine should be part of your strategy, and it’s a heck of a lot better than no backup strategy at all. But be very wary of Time Machine, because it ain’t all roses.

Time Machine, In Which There Are Dragons

Time Machine is quite clever, and uses UNIX hard links to efficiently manage disk space on the backup volume. Backups are stored in a sparse bundle file, which is a form of magic disk image that houses all backup data.

But, I swear, it is sometimes just too magic for its own good. It is decidedly not simple under the hood, and when it fails, it fails in epic fashion. See this for some examples.

I’ve had at least five backup failures that I blame on Time Machine in the past four years. In OSX Lion, one kind of failure shows up like this:

There is no recovery option provided to you. If you say “Start new backup”, it deletes your old one and begins anew. If that’s terabytes of photos/video/music/whatever, be prepared for a very long wait. Perhaps days, depending on your backup data set size, network speed, disk speed, and phase of the moon. Okay, maybe not the last part, but you never know.

And in the meantime, your backup system is gone. At this point, it is obvious that having data in at least three places is necessary. (Note that there are techniques for repairing the Time Machine backup volume. Dig out the solder and oscilloscope first, though).

You didn’t do anything, but Time Machine broke. That is completely unacceptable for a backup system.

It’s quite possible that I’m doing it all wrong. And the problems may not be Apple software errors; they may be a function of Apple+Synology, or just Synology. But that is beside the point. Any backup strategy that can fail and irrecoverably take all your data to Valhalla is a horrible strategy. I need something that cannot fail.

A better strategy, with 73% less insanity

As it turns out, I have never really cared about the save-old-versions-of-files feature of Time Machine. I have used it to recover entire volumes — twice, both during machine upgrades. Recovering an inadvertently deleted file is rather rare for me, but I suppose I do care about that feature a little bit.

Time Machine sparsebundle files are opaque to the average user. You cannot open them up, peek inside, and grab the files you need. You need the Time Machine client, and when it encounters an error with the backup file, it offers no choice but to abort and start over.

This is why I use rsync. With a little bit of Time Machine still involved for added spice. You know, just to keep things interesting.

Plus, rsync sounds cooler.

Rsync is a command-line tool, available for several platforms, and included with Mac OSX. In its simplest form, it just copies files from one place to another. But it can also remove files no longer needed, exclude things you don’t care about, and work across a network, targeting a mounted volume or a remote server that supports SSH. Which is how I use it.

The end result of an rsync backup is a mirror of your source data. Readable by anything that can read the format of the target filesystem. This part is critical. Backups are irrelevant if nothing can recover them. A bunch of files in a directory on a disk is accessible by just about everything. Time Machine sparsebundles require Time Machine, on a Mac. Files in a directory can be read by any app or OS. Thisincreases your odds of recovery by, well, a lot.

I have a few computers around the house. Our primary family computer is an iMac, and has one internal disk and two USB external disks. The internal disk has all the user folders, documents, applications, and OS files. The external drives contain photos, movies, music, etc.

My backup strategy has the internal disk backed up to my NAS using Time Machine, and the the external disks backed up to the same NAS using rsync.

Some notes:

  • The Internal HD volume is less than 100GB, and the backup executes automatically every hour.
  • I wrote simple shell scripts to automate the rsync commands. I execute the rsync scripts manually, but these are easy to automate.

I won’t go into too much detail on rsync usage (some resources that might help: 1, 2, 3), but here’s how I backup an entire external volume using rsync:

rsync -av --delete --exclude ".DS_Store" --exclude ".fseventsd" --exclude ".Spotlight-V100" --exclude ".TemporaryItems" --exclude ".Trashes" /Volumes/your-local-volume-name-that-you-want-backed-up/ user-name@backup-server:/volume-name-on-server/path-on-backup-server

 

The -av says “archive, with verbose output”. The –delete option says “get rid of anything on the server that’s no longer on my local machine” (be careful with this one). The –exclude options allow me to avoid backing up crap I don’t need. The username stuff allows me to log in to the server and perform the backup using that identity on the server.

Summary

Rsync can fail with network or disk hardware errors. Time Machine can fail with network, disk hardware, or buggy software errors. I prefer rsync for the really important stuff, and use Time Machine for the OS disk, which is relatively small and something I can recover from quickly after the inevitable Time Machine error.

iPhone camera roll bug makes photos unavailable, hack your way to a fix

TL;DR … scroll down for a solution.

What happened?

My iPhone 4 with iOS5 started failing in a peculiar way a couple of weeks ago. I could take still photos and video without any trouble, but the camera roll display (in the Photos app) would fail to show thumbnails correctly, and attempts to pull the images onto my iMac wouldn’t work. Both iPhoto ’11 and the Image Capture app on Mac OS 10.7 could see my iPhone, but said that there were 0 photos on the device.

Really, I just wanted to get my photos & video off the device. And I couldn’t. I tried lame workarounds like emailing photos to myself (worked for photos, but not large videos) and syncing to Dropbox (failed, Drobox app couldn’t read photo library … which is itself a clue).

I needed a real fix. While I don’t know what caused this problem in the first place, I have one idea. Read to the end to find out.

Symptoms

Here’s what the thumbnail view looks like:

As you can see, several blank images, and several thumbnails are duplicates. Funny thing is, you could select items and see the correct image.

In the Settings app, under General | About, the number of reported photos is off the charts. Geeks will recognize this as a rather special number.

Finally, iTunes shows some space used by “Other”. This is close to the amount of space used by my photos & videos.

Solution

After Googling a bit with no clear solution in sight, I turned to Twitter. @jkubbeaver had the same problem, and stumbled upon a solution that worked for him, documented here. In case that link vaporizes, here’s a summary:

  • Download and install iExplorer, which allows you to manage the filesystem of your iPhone.
  • Fire up iExplorer with your iPhone connected via USB cable, and, as a precaution, copy photos/videos from the iPhone’s DCIM folder to your computer. At this point, you now have the files you want, but you still need to fix the underlying problem.
  • Navigate to the iPhone’s PhotoData folder, copy the com.apple.photos.caches_metadata.plist, Photos.sqlite, and PhotosAux.sqlite files to your computer (again, as a precaution), then delete them from the iPhone’s PhotoData folder. This deletes the iPhoto camera roll database and thumbnails, but does not delete the photos themselves.
  • Reboot your iPhone, navigate to the Photos app, and let it rebuild the camera roll database automatically.

At this point, all should be back to normal. This solution worked for me.

Addendum: why did this happen in the first place?

I’m not sure yet. When I upgraded to iOS5, I did not initially enable Apple’s PhotoStream service. After a few days, I enabled PhotoStream to give it a try … and then started seeing the symptoms. Is PhotoStream to blame? Unsure, but it’s the only thing that I can remember changing between “no problem” and “problem”. YMMV.