Dark's Code Dump

Possibly useful

8mm film telecine: a software approach

There are many posts online singing the praises of 8mm telecine by scanning individual frames with a macro lens or scanner. And rightfully so – in the average case it produces better results than the alternative of realtime recording of a video projected on to a wall, projected within a film transfer box, or ‘directly’ using a special lens configuration.

With these assumptions, along with a first attempt filmed off a wall that looked mediocre at best, I went on a wild goose chase trying to build a hardware solution to what I perceived to be the greatest issues with filming off a wall: focus, frame blending and the rolling shutter effect. The focus seemed consistently worse than it could have been due to multiple variables at play: projector lens zoom, projector lens focus, camera focus, camera position, original film not always perfectly focused. To capture a flicker-free image, I found the framerate and shutter of the camera had to be configured to span potentially multiple frames of the original, which introduced a nasty blending effect. Motion appeared nice and smooth, but panning lost all detail, and ghost images occurred across transitions. Panning exhibited rolling shutter / ‘jello’ effect, vaguely similar to the tearing that results from lack of vsync, where the top half of the image would be a frame different from the bottom half.

I tried using a Sankyo variable speed projector, but it had failed to stand the test of time and had multiple issues (microswitch failures, film gate misaligned/broken). I tried focusing using a reversed lens macro configuration in conjunction with the projector lens pulled out further than usual – this resulted in a full frame in reasonable focus, but with bad chromatic aberration everywhere but the centre. Perhaps this would have worked with a macro setup with a longer working distance (max I could achieve was about 0.5cm too short), eliminating the need for the projector lens, which may have been the source of the CA. Finally, I investigated a hardware solution to run the projector off a controllable motor to advance frame by frame, but decided to save that as a last resort due to the level of hassle involved.

Eventually I came to the solution of using software to tidy up the ‘recording the wall’ approach to the same level of quality that I have seen from professional telecine with a proper machine.


  • Canon DSLR supporting Magic Lantern and video DPAF (I used a 70D)
  • Fast normal lens with minimal distortion (I used a Sigma 30mm Art f1.4 stepped down to f2.0)
  • Projector with half or variable speed control (I used a Eumig with 9fps and 18fps options)
  • A4 paper stuck on wall

I consider this a ‘software approach’ due to the lack of hardware mods or unusual hardware configurations. The only hardware involved is the bare minimum of a camera and projector.

The process is as follows:


Optimised smb.conf for Windows clients

These Samba config options have been performing fantastically well for me. It took quite a bit of research to finalise the config you see below, so figured it was worth sharing.

My setup is a Debian server with BTRFS storage, and windows clients. Requirements are:

  • Good performance, especially with lots of tiny files, complex directory trees, random writes within files, etc.
  • Preservation of all reasonable windows file attributes, such as:
    • Creation date
    • DOS attributes
    • Anything else necessary for standard desktop workload such as steam game library, documents/pictures/etc folders, git, etc.
  • Continuing the previous point: It should feel like the server is running Windows – no weird surprises.
  • No preservation of anything that allows Windows to behave in unpredictable Windows-like ways – specifically no ACL support.
  • Sane Linux filesystem permissions
    • When I browse through my windows files, they should look like a Linux home directory, no weird permissions etc.
    • Everything Windows-specific should be abstracted away into xattrs.
    • Everything should be executable on Windows, but not executable on Linux, as that is the cleanest representation of the platform difference IMO.
  • Backup-able with standard Linux tools like tar and borg (xattrs must be enabled!)
  • Correct behaviour of BTRFS features such as compression.

Without further ado:

   use sendfile = yes
   min receivefile size = 16384
   write cache size = 262144
   strict sync = no
   min protocol = SMB3
   aio read size = 1
   aio write size = 1
   allocation roundup size = 4096
   server signing = no
   vfs objects = btrfs
   create mask = 644
   acl allow execute always = yes
   store dos attributes = yes
   map readonly = no
   map archive = no
   ea support = no
   nt acl support = no

Options I specifically avoid are:

  • SO_RCVBUF=131072 SO_SNDBUF=131072 – specifically recommended to avoid by newer Samba releases.
  • case sensitive = yes – breaks too many things, performance improvement is barely noticeable.
  • strict allocate = yes – breaks BTRFS compression.

And one final note, I noticed a substantial performance improvement in my use case of SMB when upgrading from Debian 9 to Debian 10, so if you are on a lower Samba version than 4.9, consider upgrading!

Debian 10 kernel slab memory leak

I’ve faced a situation on one of my VPSes where kernel slab memory spontaneously started leaking:


In slabtop, I found the culprit to be kmalloc-64. This is fairly meaningless, but after some searching I found you can add slub_debug=U to the kernel command line. Then, you can see the source of slab allocs of this type by viewing /sys/kernel/slab/kmalloc-64/alloc_calls.

This pointed me to an issue with KVM paravirtualisation of page faults:

[20:48:53][root@kyubey][/sys/kernel/slab/kmalloc-64]# cat alloc_calls
     27 x86_vector_alloc_irqs+0xf6/0x3b0 age=1061960/1062568/1063054 pid=0-92
     15 mp_irqdomain_alloc+0x79/0x290 age=1063054/1063054/1063054 pid=0
 138245 kvm_async_pf_task_wake+0x83/0x110 age=0/474464/1062430 pid=0-1517
     31 reserve_memtype+0xb3/0x2c0 age=1060763/1062079/1063055 pid=0-273
     24 __request_region+0x6e/0x190 age=1060825/1062237/1063050 pid=1-282

My host has confirmed it is unlikely to be on their end, so I’m stumped as to where this came from out of the blue.

Anyway, a valid workaround is to add no-kvmapf to the kernel command line.

Unexplained heavy writes from MySQL killing SSD

I have recently been dealing with an upgrade from Debian 9 to 10 increasing MariaDB write load to ~20MB/s (from <0.5MB/s) on a near idle server.

Along the way I found that Linux tools to diagnose this situation are pretty bad compared to Windows. It’s nearly impossible to find out what files a process is writing to in MB/s, without turning to semi-commercial tools such as sysdig.

Sysdig showed ibdata1 being written to at ~0.5MB/s, which was odd given that figure was roughly the write load from before the upgrade. A far cry from the 20MB/s being reported by collectd, htop and my poor SSD’s smart statistics.

The culprit turned out to be BTRFS – specifically compression. For database or other small write workloads, it balloons the size of writes. I was led astray by btrfs fi defrag – this tool is not truly capable of disabling compression, despite what some posts online may say. I could see in compsize that a handful of data was still compressed, but never would have guessed that these few MB were all it took to continue to balloon the writes up!

The only approach I could find was cp -ra on /var/lib/mysql, delete the old one, and move the copy into its place.

Fix error 0x80070003 when updating Windows 10 1809 to 1903

Errors you may see:

  • Operation failed: Mount WIM file[…]
  • Error 0x80070003
  • ‘Windows 10 failed to install.’

If you have never installed Macrium Reflect including in the past, follow another guide, as there are many other possible causes. I had tried everything, and finally came across a mention that the issue stems from Macrium Reflect. This issue only occurs after you uninstall Macrium Reflect!

In regedit, go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WIMMount.

If you find the value of ImagePath to be \??\C:\Program Files\Macrium\Reflect\wimmount.sys, shake your fist at Macrium for literally breaking your system, and change the value back to the default of system32\drivers\wimmount.sys.

Debian Buster upgrade boot failure (LVM + BTRFS)

Upgraded one of my servers to Buster and was faced with an unbootable box, unable to find the rootfs. On the server in question I am running a filesystem of BTRFS RAID 1 across two independent LVM disks.

Faffed around for ages trying to diagnose, in the end had success with the nuclear option:

Create /etc/initramfs-tools/scripts/local-top/forcelvm:

   echo "$PREREQ"
case $1 in
   exit 0
. /scripts/functions

lvm vgchange -ay

btrfs device scan

Chmod it +x

Then update-initramfs -u -k all

Fix for League of Legends patch error code 004

Patching failed and we’re not sure why. Please restart the client to try patching again. (Error code 004)

If you have moved your League installation to another disk and used a symlink from the previous location to the new one, as of roughly patch 9.10, this no longer works.

Simply update your shortcuts to point directly to the new location, including the working directory, and all will be well.

GitHub does not respect GDPR in a reasonable way

Comment edit history is public (not a fan, but Facebook does this so whatever).

Deleted comments and their edit history are visible to the repository owner (yikes).

And the clincher: their support refuses to directly delete/anonymise comments under GDPR. (Mega yikes.) They claim to be a ‘processor’ and that any GDPR request has to be forwarded to the repository owner.

I’ve never heard of such a ridiculous policy and I can’t believe this is legally possible within the spirit of the GDPR. You don’t have to ask permission from a Facebook page owner to delete a comment left on their page! Somehow GitHub has managed to get themselves classed as a hosting service, not a social network. Fair enough for the git repos themselves, but god only knows how they legally wrangled that one for things like Issues.

In conclusion, think very carefully before posting anything whatsoever on GitHub. The only way to remove it is the nuclear option (delete account).

Bitlocker EDrive / IEEE 1667 Hardware SSD encryption on Dell XPS 15 9550

There is a lot of conflicting information about whether hardware encryption for Bitlocker works on the older XPS 15 laptops (9550 and probably 9560).

The answer is no, it does not work. The BIOS does not implement the required UEFI features. It fails when rebooting to apply, and when skipping the hardware test, soft-bricks the SSD.

Note I am referring specifically to Bitlocker EDrive mode (hardware encryption). Class 0 encryption also does not work on these laptops as there is no facility to set a BIOS password for NVMe disks.

Collectd disk plugin produces no output

I spent a while scratching my head over the collectd ‘disk’ plugin producing no output whatsoever on a Debian 9 server. No log output (other than successful load), no rrd files, nothing.

Frustratingly, debug logging in collectd is compiled out by default, and compiling collectd from debian source packages requires an insane number of dependencies, so I avoided that route.

Instead through analysing the source and history of the disk plugin I came across this commit. The issue turned out to be simple – in the backported kernel I was using, /proc/diskstats has a tonne more fields than the default 4.9 kernel, and this was causing the disk plugin to fall over silently.

It’s a ballache to do anything about this situation on Debian 9, assuming the backported kernel is essential. If you hit it then best just wait for Buster.