Applied Magic

Slacker Reset

I was psyched to get my first PC in the autumn of 1992. It was what they used to call an “IBM clone,” as IBM was the only Intel-based PC back then and there was no Dell or big name home computer resellers. There was no real internet yet either, but to me this offline PC was 33MHz of pure joy. It was my senior year at Colorado State and I logged more hours playing games on the computer than writing papers for school. Thus the PC earned the name “Slacker,” and this name would be passed down to future incarnations of my primary gamer rig in the decades to come.

Now, almost thirty years later, the current incarnation of Slacker ran a quad-core 3,300 MHz Intel CPU with a decent graphics card. It was a modest rig by contemporary gamer standards, but still enough to run the games I played.

Last weekend I got the idea that wanted to wipe and reload Slacker. There were no glaring technical issues, no malware or spyware known to me (the PC had never seen a single bit of downloaded data from nefarious web sites and I was ultra-paranoid about PC and browser security). It was just that I had been using the same build of Windows for a really long time.

During the move from Japan, I had disassembled Slacker and stored the parts in static-free bags, wrapped in tape and neatly packed into a suitcase. Upon arriving in the States I assembled all the parts in a new case, and by some miracle everything booted up and worked fine. (See featured image for Slacker resurrection in November of 2019).

Slacker remained functional, but I was still using the same OS and some stuff was a bit quirky, like the fact that it took at least ten minutes to load one of my favorite games, Total War Shogun 2. This was an older game that should’ve loaded in seconds. There was also a persistent problem with a mouse driver that aroused my fury when it spun out of control and randomly scrolled through my choice of weapons during a critical moment in a first person shooter game.

Plus, I just wanted a clean slate, the feeling of having the disks wiped and the underlying OS fresh and trim, which was perhaps the same emotion as loving that smell of new car interior.

Bottom line: Slacker needed a fresh start.

Windows included an option to “reset this PC,” with a choice of keeping data and reloading the OS, or nuking everything from orbit. My choice was the latter, of course, because what idiot would keep personal data on the system drive? I maintained a few elaborate scripts that kept my data secure and squared away, but there were a couple things to consider before blowing away the system, mainly regarding licensing. Licensing wasn’t the most exciting topic in the world. Most people threw money at it to make the problem go away (i.e., the software subscription route).

Me, I was still holding on to the more economical option of using the same Windows license I had been using for more than a decade (that had started with a FREE copy of Windows 7 snagged off Technet before Microsoft shut the doors on that program, and had been carried over through subsequent upgrades). Would my ancient licensing legitimacy get dropped in the reset? Microsoft wasn’t very forthcoming about what went on under the hood in this reset (as usual). So, to be safe I backed up the system disk to a USB drive using a great tool called Windows 7 Backup, still included in Windows 10.

There was also the issue of Microsoft Office. I was still using the last purely desktop-based version of Office (2013) because I didn’t care for the idea of paying Microsoft a minimum of $100 per year for the rest of my life, just for the privilege of occasionally opening a spreadsheet or using Word. Plus, Office hadn’t improved in a decade or so. It didn’t need improvements. It was done a long time ago. Microsoft Office used to be desktop software that seemed a bit pricey even twenty years ago, like $300 or more. Now, with a subscription to Office 365, it was at least $3,000 over time, considering I’d be paying for at least another 30 years. So, I double-checked my ISO library to confirm I had Office 2013 and the license key handy. (I had also grabbed this off Technet … for free!)

And because I’m a super-paranoid neat freak when it comes to everything, especially computers, I also decided to take this opportunity to zero-out and triple-wipe (by DoD standards, of course) EVERY DISK in use, including (3) half-terabyte SSD disks in the PC, and (3) two-terabyte external USB drives. Two of these external USB drives were a mirrored, secondary copy of all my digital keepsakes, making the operation a tricky juggling act. (The primary copy was in the cloud, a private and secure solution offered by a Swiss company that matched my level of paranoia.) But despite its trickiness this kind of activity was nothing for any decent IT person, and I had been in the game for a while.

It went like this:

  1. Backed up the system disk to USB drive 0, and noted the size was around 90 GB used space. (This backup would not be needed, thankfully.)
  2. Uninstalled all games from Steam that were no longer played by me or my boys.
  3. Backed up local saved game files for games we still cared about. (This turned out to be not needed, of course, as Steam saved games to the cloud; but I wanted to be 100% sure not to lose anything.)
  4. Uninstalled Steam.
  5. Triple-wiped internal SSD disk 1 (games).
  6. Uninstalled cloud storage service agent.
  7. Confirmed backups to Swiss cloud and USB.
  8. Triple-wiped internal SSD disk 2 (backup).
  9. Ran the Windows reset with nuke-from-orbit option. It did wipe the OS partition on SSD disk 0, but did not wipe the two tiny reserved partitions. This was probably how it retained Slacker’s core consciousness … and the license key, too?
  10. Triple-wiped internal SSD disk 1 again.
  11. Triple-wiped internal SSD disk 2 again.
  12. Set up new Windows environment, which didn’t seem as much of a hassle as in years past. Really it was just installing Windows updates, Firefox, the video card driver, Steam, and the Swiss cloud storage agent. Later I’d install Office 2013.
  13. Created Steam library on internal SSD disk 1 (games).
  14. Started synch of cloud data to designated vaults on internal SSD disk 2 (backup).
  15. Made backup of USB drive 1 to temporary location on internal SSD disk 2 (backup).
  16. Triple-wiped USB drive 1.
  17. Mirrored the designated data vaults on internal SSD disk 2 to USB drive 1.
  18. Triple-wiped USB drive 2.
  19. Mirrored USB drive 1 to USB drive 2.
  20. Triple-wiped USB drive 0 when it was confirmed the new Windows install was functional and was confirmed that I did not need a new Windows license key!

All this took a couple hours, but as mentioned, it wasn’t too bad. I was loving that fresh, new PC smell.

As an added bonus, Total War Shogun 2 now loaded in ten seconds instead of ten minutes, and that infuriating mouse problem went away.

Applied Magic

Multimedia Archiving Project – Intro

Wouldn’t it be awesome if there was an easy way to catalog and control the ever-rising deluge of photos and videos we generate with our devices, a system of organizing that could be transferred to future family members for safe-keeping? What if this system had the following traits?

  1. locally-controlled (by you)
  2. decentralized (resilient)
  3. platform-independent
  4. with a standardized file structure
  5. and a standardized file naming scheme
  6. that is both effortless
  7. and flexible

Well, that would be awesome, indeed.

It so happens I have such a system. It achieves the first five of the above traits, but it’s not yet effortless (if there is such a thing) or flexible. Without me the system falls into disarray. This post isn’t exactly a life hack – not yet, anyway. It’s the first of what will be several posts on this project as I get closer to making it more flexible and easier for others to use. With this post I wanted to explore the reasons for such a system, and to illustrate the general idea.

Ten years ago my dad gave me a box of photos and slides, which I scanned and integrated into my family’s multimedia mess. This effort began a home-grown archiving system that would come to be known as the Multimedia Archive Project (MAP). For me it’s a workable solution, still evolving today.

My ultimate long-term goal is to establish a system (more a protocol than an actual set of tech) that is easily transferable to my kids and subsequent generations. A decade later I’m in a holding pattern, still looking for technology that could suit my needs. Here’s some more details about what I mean by the above traits and why they’re important to me.

What is “locally-controlled”? I want the primary location of my family’s multimedia files to remain in my hands, so to speak. I’m not contributing to the oceans of photographic knowledge that an artificial intelligence uses to shape the world in ways I don’t see fit. This might seem paranoid now, but the world is starting to understand “free” services for individuals have big costs to society as a whole. It’s very important that I maintain control.

“Decentralized” just means it’s impossible to lose data. This part isn’t exactly effortless. It requires discipline and planning that most people aren’t willing to do. The Multimedia Archiving Project is backed up in the same system I use to back up all the data in my household, which includes consolidation and copies made to mirrored USB drives, a NAS, and a cloud service based in Switzerland that is a stickler for General Data Protection Regulation (GDPR) rules.

“Platform-independent” is another cornerstone of this project. I don’t want to be locked into any single app, or depend on one company’s services. When I started this effort ten years ago the big cloud services were making it difficult to switch platforms. They’re a little nicer now, but to some degree this is still true.

Apple offers a great all-in-one photo archiving solution, and they’ll no doubt be around for decades to come. I’d be the first to recommend Apple to anyone who doesn’t have the discipline or technical chops to handle a DIY solution, based on their track record of quality software (iTunes for Windows not withstanding) and data privacy. Still, I prefer platform-independence. My family and I have some Apple devices, but we don’t have a Mac. What if I put my trust in a company who changes the rules twenty years down the road in a way that is unethical, inconvenient, and-or too expensive for me?

At the opposite end of the spectrum, I’d rather delete everything than trust a mind-control advertising platform like Facebook or Google with my family memories. Last year we went skiing with another family in beautiful Niseko, Japan. The other dad and I were on the mountain together one afternoon and he took a video of me as we skied down the slopes. I thought it would be an awesome video, with the sun in the right position and the spectacular scenery. And it was! The only problem was it had been live-streamed to Facebook and I didn’t have a Facebook account. Never mind me. What if the photographer wanted to preserve this video – or any media – and pass it down to his kids? They’d need Facebook accounts, too. This illustrates the importance of platform-independence. It’s the freedom to never be locked down to a proprietary system that defines how you can use your own stuff.

There are some very positive trends in digital identity that could work in my favor as the decades unfold (see previous post). Bottom line, I want the flexibility of moving my family memories securely and safely, with the maximum privacy levels, whenever I want.

The “standardized file structure” and “file naming scheme” are the coolest features of the system. They’re inspired by ISO standards. This gets into how this system works.

How does this thing work?

First, there are rules, because every system has rules. Fortunately most of the rules are enforced by code, but the first one must be observed by humans: NEVER MODIFY ORIGINALS.

The second rule is there is one and only one destination path for any given source device, a file folder in the ORIGINALS directory. For example, there is a folder for all the originals backed up from my wife’s iPhone, a folder for my camera photos, a folder for our Gopro, and so on. These devices and paths are configured in an XML file. This system runs on Windows, so I use PowerShell. In the future I might go with Linux and Python.

The process begins by running a script to “add new files to archive,” which reads the XML file for source and destination paths, checks to see if the devices in question are plugged into the system, and if so compares the latest photo and video files on the device with what’s already in the archive. If there’s new stuff then it copies it to the destination folder. I run a separate script to rename the files in a standard format so that anyone can take one look at the file name to know the date it was created, by whom, and where (all this data is available in the metadata of standard media files). A five-digit sequence number is tacked to the end of the file name. Ten years ago I never thought I’d have more than 99,999 files per device, but who knows? My wife is approaching 10,000 photos and videos now after five years with one iPhone (and these are the files remaining after she deletes stuff from her phone).

Since rule number one is NEVER MODIFY ORIGINALS (renaming doesn’t count as a modification, as it does not change the “last modified” timestamp), I maintain a separate directory for “COLLECTIONS,” which are basically photo albums of certain events or seasons. This is a manual effort and probably always will be. I don’t have the AI at my disposal to magically identify people, places, and events to assemble a photo album on the fly.

When the files are copied, updated, and renamed, I then kick off the backup script (basically a fancy Robocopy) to replicate the changes to the various backup locations, including the folder to sync with the cloud service.

In the future I might keep this basic system intact but expose a portion of it to a paid AI service to assist with categorizing, facial-recognition, tagging, and the like.

In days of old, family memories might be preserved in the form of hard-copy photographs in a shoe box. Back then, the problem was keeping this single point of failure safe from fires and floods. Now, the problem is we have too much stuff. Some intervention is necessary, and this system works for me. As for “effortless,” I’m not sure I’ll ever completely reach this goal. Maybe the point of an archiving system is that is should require some effort, otherwise how do we decide how we’re represented by future generations?