Dark's Code Dump

Possibly useful

Windows Dynamic Disk vs Storage Spaces vs Intel RST for RAID 1

Disks are 2x Western Digital Red 4TB. All configurations are RAID 1. (Technically Storage Spaces Mirror isn’t true RAID 1, but it’s the closest possible for this comparison.)
Disregard the varying partition sizes – all tests are at the start of the drives.

Individual disk

Dynamic Disk

Dynamic Disk

Seems to be good for performance, and is more portable than any hardware or fake RAID. Contrary to some outdated posts elsewhere, it does distribute reads across both drives. However it does not distribute sequential reads, so performance is not as great as it could be for some workloads.

The major drawback is how easily resync is triggered and how much it destroys performance for real-world read loads until complete. For example I loaded Divinity 2 while resyncing and it took minutes as opposed to seconds. Any time you happen to BSOD or otherwise improperly shutdown, or even hibernate (including fast startup) it triggers a resync.

Partition management is limited – the only software I could find capable of editing an existing Dynamic Disk setup was EaseUS, and when tested it froze attempting to resize an empty partition and hosed the RAID. Additionally if you have more than one partition on a dynamic disk RAID, resyncing occurs on each partition simultaneously, so on standard hard disks it destroys resync performance to have more than one partition!

Overall good performance and well-supported for future migrations, but a no-go for more than one partition, large disks, hibernation or PCs without flawless stability.

Storage Spaces

Storage Spaces (default Thin provisioning)

For the longest time I’ve been using Storage Spaces with zero data issues, but the poor performance of gaming workloads pushed me to look for alternatives. The preallocation done by Steam and Fortnite updates is extremely slow under Storage Spaces compared to bare disks(!) or any other type of RAID I’ve tried.

The benchmark results mask just how slow certain workloads can be – anything that involves reading/writing to the same file (e.g. patching) or allocating a large file using a non Windows API approach is dog slow. (OTOH fsutil file createnew is fast.)

Storage Spaces (fixed provisioning)

That said, Storage Spaces still manages to outperform a single disk slightly for reads, and even for writes, with the latter likely due to the abstraction reordering the random writes produced by CrystalDiskMark into something more optimal.

I’ve seen little discussion of Fixed provisioning of Storage Spaces online – it’s hidden behind arcane powershell commands. As it supports normal defrag (which implies a more natural on disk format) I was hoping for performance to exceed thin provisioned storage spaces, but it is actually slower across the board. There doesn’t seem to be any reason to use fixed provisioning over the default thin provisioned mode.

Speaking of on disk formats, it scares me slightly that the contents of each disk isn’t a carbon copy of the other.

Compared to dynamic disks, partition management is good – you can resize and add/remove partitions at will with even more flexibility than an ordinary disk. Tools support is good as each partition is presented to the system as a Basic disk.

The resyncing issue of Dynamic Disks doesn’t exist at all in Storage Spaces – throughout countless BSODs, resets, power losses, etc. I’ve not once seen Storage Spaces do any kind of resyncing, nor have I experienced any issues with my data stored on it.

Overall good support and a solid choice if redundancy and data protection are your aims, but the performance is lacking

Intel Rapid Storage Technology

I never thought I’d find myself considering fake RAID, but given I’m currently using a Supermicro server motherboard in my desktop with a special ‘enterprise’ incarnation of Intel RST, I figured I’d give it a shot. Note: I have reason to believe the performance of RSTe (as opposed to RST on desktop-grade motherboards) is improved, especially for RAID 1, so take these numbers with a pinch of salt.

Intel RSTe

The benchmark results for read performance eclipse the other options – even sequential reads are distributed across both disks. Writes are slower, but this is with all the dangerous write caching options disabled (I have no UPS). I can confirm these performance figures in real world performance – everything is subjectively faster than the other configurations.

I’m mildly concerned about what happens when I upgrade my PC and/or my motherboard fails, as my future plans lie with AMD. I put my mind at ease slightly when I discovered there is an option (unique to RAID 1) on RST that can turn the first disk back into a standard non-RAID disk without rewriting the data, so the on disk format must be close to an individual disk. Seeing the amount of recovery software available as well as a pure-software implementation in Linux also gives me hope I’m not digging my data into a hole. I actually feel happier with Intel RST than I would do with a hardware RAID card in this regard – at least it will work on any Intel board rather than one particular RAID controller.

This is good old fashioned fake raid, so partition management and tools support is flawless – as far as software is concerned it is a single disk.

Like Dynamic Disks, resyncing is an issue in the case of unclean shutdowns/BSODs/etc. However it is a nicer implementation – it is not triggered by hibernation/sleep, and it can be paused and resumed by hibernation or shutdown, as opposed to dynamic disks which will restart. It also does not attempt to resync multiple partitions at once, so it is a relatively fast linear operation. System performance impact during resyncing is roughly the same as dynamic disks – i.e. significant.

Overall performance is excellent, software support is flawless, future thinking is slightly better than true hardware RAID but behind the other software options.

Final thoughts

There’s no one-size-fits-all solution, but I’ve found myself doing a complete 180 from ‘lol fake raid who uses that’ to actually recommending Intel RST above other options for Windows.

Dynamic Disks work well until you inadvertently trigger a resync, then you spend the next day being reminded of how bad they are. The design goals behind Storage Spaces – archival and flexibility – clearly come at the cost of performance.

I’ve ignored proper RAID cards, but ultimately until Microsoft get their act together and create something on the same level of MD or BTRFS, hardware remains king. I wouldn’t be surprised if there was indeed a financial reason why software RAID is slower on Windows.


Android Pie battery drain

I’ve been experiencing chronic battery drain since updating to Android Pie, attributed to Android System, Android OS, Google Play Services.

I’ve finally found the culprit: the Pebble app. Replaced with Gadgetbridge and all is well.

The cause seems to be abuse of Google Play Services and of WebView (‘sandboxed_process0’) causing significant drain while the phone is awake and contributing to drain while locked. Probably underpinning features nobody asked for, like the Javascript SDK, PebbleKit, location-based weather, Pebble internet connectivity, etc. Gadgetbridge excludes all of that nonsense by design or as an option, don’t know why I didn’t switch sooner!

And as a handy side effect, phone calls no longer show up as from Unknown!

VW fault code P188900 / P1889 / P1888

P188900 – Coolant shut-off valve – Short circuit to ground/open circuit, Coolant shut-off valve -N82 Short circuit to ground/
open circuit

Getting this recurring fault code reported by CarPort on a Mk7 Golf. Dealership is adamant there are no faults reported via their diagnostic system, so presumably this is a false positive. I’m not getting any symptoms or warning lights whatsoever.

If you’re in the same boat, don’t bother chasing this up and don’t panic from what you’ll see on Google wrt coolant leaks destroying wires.

Veeam backups failing with network errors

I use Veeam Free and recently started getting network errors when trying to perform any backups. My destination is a linux box via samba, to an SMR disk with BTRFS – which can be quite unpredictably slow.

Errors included:

Error: Agent: Failed to process method {Transform.Patch}: An unexpected network error occurred. Failed to flush file buffers. File: [\\Redacted\Backup Job 2018-07-04T153041.vbk].

Error: An unexpected network error occurred. Failed to write data to the file [\\Redacted\Backup Job 2018-10-12T140141.vbk]. Agent failed to process method {Stg.RemoveOrphanedSessionKeys}.

Full backup file merge failed Error: Agent: Failed to process method {Transform.Patch}: An unexpected network error occurred.

Full backup file merge failed Error: An unexpected network error occurred. Failed to rename file from [\\Redacted\Backup Job.vbm_tmp] to [\\Redacted\Backup Job.vbm]. –tr:Error code: 0x0000003b –tr:FC: Failed to rename file from [\\Redacted\Backup Job.vbm_tmp] to [\\Redacted\Backup Job.vbm]. –tr:Failed to call DoRpc. CmdName: [FcRenameFile].

Very simple solution, in admin powershell:

Set-SmbClientConfiguration -SessionTimeout 600

How to actually reduce battery drain on Android

This assumes you’ve tried all the normal methods of reducing drain – apps like BetterBatteryStats, Wakelock Detector, etc, the built-in battery stats, trimming permissions, using Pie background restrictions, etc.

I very rarely see this mentioned anywhere when people are trying to track down mysterious Google Play Services/Android OS/Android System drain – most people seem to resort to a factory reset, which won’t fix the problem when the issue lies in dodgy apps.

The fix is really simple:

(more…)

Print tar label without reading entire tar archive

GNU Tar lacks an option to print the tar label and exit immediately – it can only print the label as part of reading the whole archive. This works ok-ish for hard disks where ctrl+c is instant and without side effects, but is unsuitable for tape drives.

The following simple perl one-liner prints the label from a tar archive passed in stdin:

cat my.tar | perl -e 'read(STDIN, $header, 100); print(unpack("Z100", $header)."\n");'

An alternative, hackier, approach is discussed in my post on LTO tape drives: https://darkimmortal.com/2018/10/adventures-with-lto-tape-for-home-casual-use/

 

How to merge old VirtualBox snapshots into single VDI

I was recently faced with a situation where I had a the data of an old VirtualBox VM including .xml file, .vdi/.sav snapshots and main .vdi hard disk from 2012 that I wanted to merge into a single .vdi.

Crucially these files were from another machine and not registered/present in any way in my active copy of VirtualBox.

There doesn’t seem to be a clear set of instructions for doing this online, but the process is:

  1. VBoxManage showhdinfo [harddisk.vdi]
    • Note ‘showhdinfo’ is, confusingly, not just showing the info but also registering the disk so VirtualBox can find it later
  2. VBoxManage showhdinfo [snapshot.vdi] for each of the snapshots in order of oldest to newest
  3. VBoxManage clonehd [newest-snapshot.vdi] [output.vdi]

Adventures with LTO Tape for Home/Casual Use

I’ve spent the past while buying and experimenting with some old LTO gear as a secondary backup solution besides backing up to both local disks and cloud. I chose LTO because:

  • My upload speed is trash so the only way I can hope to do a full off-site backup is via physical media
  • With LTO4, it works out cheaper than buying about 2-3x 2+TB external hard disks, especially if you go SCSI instead of SAS.
  • I have the option of taking tapes out of rotation and storing them long-term. With hard disks this is risky and expensive.
  • Good for air hand luggage
  • It’s old school cool

My learnings and notes follow…

(more…)

eBay support literally encouraging fraud

Goes without saying that I’m returning the item manually and not following their advice!

02:34:36 UTC (me)

Hello, I recently opened a return and the seller has sent me a refund before I had a chance to print the postage label, and now the option to print a postage label has disappeared. How can I get the postage label to return the item with?

02:35:43 UTC (ebay support)

Alright, thank you for the confirmation and sharing the details with me. I can understand that you are concerned about the return case in which you received the item and refund both. I will look into it and help you with best possible resolution from my side.

02:36:33 UTC (ebay support)

Just to confirm, are you referring to item “redacted”?

02:36:47 UTC (me)

yes that's the one

02:38:06 UTC (ebay support)

I have checked the case details and found that case is closed as seller has issued the refund to you. Now, you are not required to take nay further action in this. You can keep the item or you can return it by communicating with seller. However, the case is closed so no further action is required.

^ What the literal fuck

(more…)

VW Composition Media M3U playlist bug

Spent the best part of 2 days trying to figure out why the composition media system (same probably applies for discovery media system) in a MK7.0 VW Golf was failing to read M3U playlists, with the error “No playable files found”.

Tried every combination imaginable of file paths, separators, line endings, and even went as far as writing a script to generate extended m3u data from a plain m3u, but nothing made any difference.

The fix turned out to be very simple: have more than one folder on the SD card with MP3 files in it. The software must have a special case for all MP3 files in one folder that (probably unintentionally) breaks playlists. Even a single mp3 file in a second folder is enough to kick it into life.

Once this fix is in place the M3U support is pretty comprehensive – plain or extended and a variety of line endings and path separators all work fine.