I recently came across Arnaud Chéritat's polynomial mating movies and just had to try to recreate them.

If \(p\) and \(q\) are in the Mandelbrot set, they have connected Julia sets for the quadratic polynomial functions \(z^2+p\) and \(z^2+q\). If they are not in conjugate limbs (a limb is everything beyond the main period 1 cardioid attached at the root of a given immediate child bulb, conjugation here is reflection in the real axis, the 1/2 limb is self-conjugate) then the Julia sets can be mated: glue the Julia sets together respecting external angles so that the result fills the complex plane (which is conveniently represented as the Riemann sphere). It turns out that this mating is related to the Julia set of a rational function of the form \(\frac{z^2+a}{z^2+b}\).

One algorithm to compute \(a\) and \(b\) is called "slow mating". Wolf Jung has a pre-print which explains how to do it in chapter 5: The Thurston Algorithm for quadratic matings.

My first attempts just used Wolf Jung's code and later my own code, to compute the rational function and visualize it in Fragmentarium (FragM fork). This only worked for displaying the final limit set, while Chéritat's videos had intermediate forms. I found a paper which had this to say about it:

On The Notions of Mating

Carsten Lunde Petersen & Daniel Meyer

5.4. Cheritat moviesIt is easy to see that \(R_\lambda\) converges uniformly to the monomial \(z^d\) as \(\lambda \to \infty\). Cheritat has used this to visualize the path of Milnor intermediate matings \(R_\lambda\), \(\lambda \in ]1,\infty[\) of quadratic polynomials through films. Cheritat starts from \(\lambda\) very large so that \(K_w^\lambda\) and \(K_b^\lambda\) are essentially just two down scaled copies of \(K_w\) and \(K_b\), the first near \(0\), the second near \(\infty\). From the chosen normalization and the position of the critical values in \(K_w^\lambda \cup K_b^\lambda\) he computes \(R_\sqrt{\lambda}\). From this \(K_w^\sqrt{\lambda} \cup K_b^\sqrt{\lambda}\) can be computed by pull back of \(K_w^\lambda \cup K_b^\lambda\) under \(R_\sqrt{\lambda}\). Essentially applying this procedure iteratively one obtains a sequence of rational maps \(R_{\lambda_n}\) and sets \(K_w^{\lambda_n} \cup K_b^{\lambda_n}\), where \(\lambda_n \to 1+\) and \(\lambda_n^2 = \lambda_{n-1}\). For more details see the paper by Cheritat in this volume.

What seems to be the paper referred to contains this comment:

Tan Lei and Shishikura’s example of non-mateable degree 3 polynomials without a Levy cycle

Arnaud Chéritat

Figure 4The Riemann surface \(S_R\) conformally mapped to the Euclidean sphere, painted with the drawings of Figure 2. The method for producing such a picture is interesting and will be explained in a forthcoming article; it does not work by computing the conformal map, but instead by pulling-back Julia sets by a series of rational maps. It has connections with Thurston's algorithm.

I could not find that "forthcoming" article despite the volume having been published in 2012 following the 2011 workshop, so I emailed Arnaud Chéritat and got a reply to the effect that it had been cancelled by the author.

My first attempts at coding the slow mating algorithm worked by pulling back the critical orbits as described in Wolf Jung's preprint. The curves look something like this:

A little magic formula for finding the parameters \((a, b)\) for the function \(\frac{z^2+a}{z^2+b}\):

\[a = \frac{C(D-1)}{D^3(1-C)}\] \[b = \frac{D-1}{D^2(1-C)}\]

where \((C,D)\) are the pulled back \(1\)th iterates. This was reverse-engineered from Wolf Jung's code, which worked with separate real and imaginary components, with no comments and heavy reuse of the same variable names. I'm not sure if it is correct but it seems to give useable results when plugged into FragM for visualization:

I struggled to implement the intermediate images at first: I tried pulling back from the coordinates of a filled in Julia set but that needed huge amounts of memory and the resolution was very poor:

Eventually I figured out that I could invert each pullback function into something of the form \(\frac{az^2+b}{cz^2+d}\) and push forward from pixel coordinates to colour according to which hemisphere it reached:

I struggled further, until I found the two bugs that were almost cancelling each other out. The coordinates in each respective hemisphere can be rescaled, and thereafter regular iterations of \(z^2+c\) until escape or maximum iterations could be used to colour the filled in Julia sets expanding within each hemisphere:

After that it was quite simple to bolt on dual-complex-numbers for automatic differentiation, to compute the derivatives for distance estimation to make the filaments of some Julia sets visible:

I also added an adaptive super-sampling scheme: if the standard deviation of the current per-pixel sample population divided by the number of samples is less than a threshold, I assume that the next sample will make negligible changes to the appearance, and so I stop. This speeds up interior regions (which need to be computed to the maximum iteration count) because the standard deviation will be 0 and it will stop after only the minimum sample count. I also have a maximum sample count to avoid taking excessive amounts of time. I do blending of samples in linear colour space, with sRGB conversion only for the final output.

Get the code:

git clone https://code.mathr.co.uk/mating.git

Currently about 600 lines of C99 with GNU getopt for argument parsing, but I may port the image generation part to OpenCL because my GPU is about 8x faster than my CPU for some double-precision numerical algorithms, which will help when rendering animations.

]]>Over xmas I played around with LaTeX to see about making a book of all my code. The amount of code surprised me, around 7000 pages, so I don't think I'll print it after all (print on demand would be around £1.50/volume + 2p/page for paperback, significantly more per volume hardback). It was suggested to me that microfilm would be best for long term archival, so maybe I'll think about that - it costs around 2p/page for 16+ rolls from digital, though I don't know how many pages a roll contains.

The main issue I faced when converting the code was that the listings package does not support UTF-8 properly. Yaxu posted a clean solution on stack exchange, but it involves listing each extended character with its LaTeX replacement, and Unicode has many code points. I have not tried this yet, instead using iconv to transliterate to Latin1 and hoping for the best (I also needed some mild sed to transliterate some Latin1 characters that are only valid in maths mode).

My experiments are in my code hosting, with configuration files for most of code.mathr.co.uk and some third-party software (mainly numerical code that I want to read):

git clone https://code.mathr.co.uk/source-codes.git

Future work may evolve into adding hyperlinks within each PDF, so that you can click on the usage of a symbol to jump to its definition; I'd probably need to use something other than Bash to implement that, maybe Haskell has libraries for enough languages, or maybe Python (though I don't like it).

I uploaded some snapshots of my repositories.

]]>I don't have a smartphone, so it wasn't something I noticed, but my previous blog layout with sidebar navigation on the left was not so good for mobile devices with small portrait screens. So I moved the navigation links below the main article/index content, with a link from the page top header. This also has the bonus of making the relevant part of the page load quicker, with about 1.3kB of HTML before the article starts instead of 8.3kB previously. I also added a meta tag that makes it display at a better size by default on mobile devices:

<meta name="viewport" content="width=device-width, initial-scale=1" />

I don't know how/why that tag works, but I saw it recommended all over the place, so maybe it's just how things are broken by default these days and you have to jump through hoops to fix them.

I also added a link back to my website home page from the blog header, and made a few small tweaks to the style sheets (thicker borders around the interesting part of the page).

These changes are spread between two repositories, the public cmcms repository with shell script and XSL style sheets for generating the HTML pages, and a private one with the source code for all the pages and static assets like style sheets and images. I did update the examples in the cmcms repository, but if you're using cmcms with your own site then do note that if/when you upgrade you'll have to fix your CSS to make it look sensible.

]]>My first real experience of computers was with an Amiga A500+, a home computer from the early 1990s with a 7 MHz processor. One thing of note about Amiga hardware is the floppy disk drive controller: more advanced than PC hardware in some ways, Amiga drives can read/write 720kB PC format disks with the right software, but PC drives can't handle 880kB Amiga format isks. One traditional way of transferring files is a serial link, but modern PCs don't come with native serial ports, and buying a USB adaptor and a serial cable seemed like too much effort, when I already have a USB sound card with a MIDI interface for my modern PC and a MIDI interface for my Amiga, and MIDI cables. So I figured I'd try to write some software to transfer files using MIDI.

MIDI has a message protocol with the highest bit of 8 set to 1 for status bytes, and data bytes having the top bit 0 leaving 7 bits for the payload. System Exclusive messages provide a way to sidestep the music-specific parts of the protocol (note on and off messages, etc) and have arbitrary packets of 7-bit data bracketed by some metadata. I'm using the non-commercial SysEx manufacturer ID 0x7D, which should not be in released products apparently. Don't know if it's worth trying to register something, anyway this is all very preliminary and experimental. Maybe it will matter more if you have a lot of other MIDI devices connected.

Linux's ALSA layer has a MIDI driver, and the amidi tool can send and receive formatted SysEx files fairly straightforwardly:

Command line using Linux ALSA tools to send file over MIDI:

amidi -p hw:2,0,0 -s input.syx

(NOTE: replace the device with your own MIDI hardware.)

Command line using Linux ALSA tools to receive file from MIDI:

amidi -p hw:2,0,0 -r output.syx

(NOTE: press Ctrl-C to terminate once the transfer is complete.)

ASCII text data is natively 7-bit so that can be packed into SysEx quite easily, but 8-bit binary data needs repacking. I went for a simple but wasteful protocol, splitting each 8-bit input byte into two 4-bit nibbles and storing them in adjacent 7-bit output bytes. This wastes 3/8 bits, but so far I transferred only small files so the slowness was acceptable. I wrote a small C program to format ASCII as 7-bit or binary data as nibbles into MIDI SysEx:

#include <stdio.h> #include <string.h> int main(int argc, char **argv) { if (argc != 2 || (strcmp("ascii", argv[1]) && strcmp("nibbles", argv[1]))) { fprintf(stderr, "usage: %s (ascii|nibbles)\n", argv[0]); return 1; } int ascii = ! strcmp("ascii", argv[1]); fputc(0xF0, stdout); fputc(0x7D, stdout); int c; while (EOF != (c = fgetc(stdin))) { if (ascii) { fputc(c, stdout); } else { int hi = (c >> 4) & 0xF; int lo = c & 0xF; fputc(hi, stdout); fputc(lo, stdout); } } fputc(0xF7, stdout); return 0; }

A better approach might be to pack binary more tightly, 8x 7-bit output bytes for 7x 8-bit input bytes, but the code is quite a bit more complicated.

At the other (Amiga A500+) side of the link I needed some software to receive and decode the incoming MIDI SysEx messages. I briefly tried an OctaMED Pro 4 magazine coverdisk edition, but that couldn't save SysEx and in any case I would need to strip off 2 bytes at the start and 1 at the end to get the raw ASCII, and further process to get binary from nibbles. Luckily I eventually found a copy of Blitz Basic 2, and wrote a small program to receive and extract MIDI SysEx as ASCII:

WBStartup ser.l = OpenSerial("serial.device", 0, 31250, 144) SetSerialBuffer 0, 100000 out.l = WriteFile(1, "RAM:out") FileOutput 1 Repeat c.w = ReadSerial(0) Until c.w = $F0 c.w = ReadSerial(0) If c.w = $00 c.w = ReadSerial(0) c.w = ReadSerial(0) EndIf c.w = ReadSerial(0) While c.w <> $F7 Print Chr$(c.w) c.w = ReadSerial(0) Wend CloseFile 1 DefaultOutput CloseSerial 0 End

(TODO: error checking for OpenSerial and WriteFile.) This was more complicated and time consuming than necessary as my Amiga A500+'s E key is broken. I had to find the a text file with E and e characters to copy and paste for each occurrence. Then I could write the program to deal with binary nibbles on my Linux machine with working keyboard and more advanced text editor (though no Blitz Basic 2 online help), and transfer it over MIDI as ASCII. Here's the Amiga Blitz Basic 2 program to receive and extract binary data from MIDI SysEx as nibbles:

WBStartup ser.l = OpenSerial("serial.device", 0, 31250, 144) SetSerialBuffer 0, 100000 out.l = WriteFile(1, "RAM:out") FileOutput 1 Repeat c.w = ReadSerial(0) Until c.w = $F0 c.w = ReadSerial(0) If c.w = $00 c.w = ReadSerial(0) c.w = ReadSerial(0) EndIf c.w = ReadSerial(0) While c.w <> $F7 b.w = (c.w & $F) LSL 4 c.w = ReadSerial(0) If c.w <> $F7 b.w = (c.w & $F) | b.w Print Chr$(b.w) c.w = ReadSerial(0) EndIf Wend CloseFile 1 DefaultOutput CloseSerial 0 End End

So now I can transfer arbitrary data from Linux to Amiga over MIDI, the next step was going the other direction. I wrote two small programs to format ASCII and binary as nibbles, and send over the MIDI serial link, ASCII first:

WBStartup ser.l = OpenSerial("serial.device", 0, 31250, 144) SetSerialBuffer 0, 100000 in.l = ReadFile(1, "RAM:in") WriteSerial 0, $F0 WriteSerial 0, $7D While NOT Eof(1) char.b = 0 ReadMem 1, &char.b, 1 WriteSerialMem 0, &char.b, 1 Wend WriteSerial 0, $F7 CloseFile 1 CloseSerial 0 End

and binary nibbles:

WBStartup ser.l = OpenSerial("serial.device", 0, 31250, 144) SetSerialBuffer 0, 100000 in.l = ReadFile(1, "RAM:in") WriteSerial 0, $F0 WriteSerial 0, $7D While NOT Eof(1) char.b = 0 ReadMem 1, &char.b, 1 c.w = char.b & $FF WriteSerial 0, (c.w LSR 4) & $F WriteSerial 0, c.w & $F Wend WriteSerial 0, $F7 CloseFile 1 CloseSerial 0 End

Finally I wrote a small Linux C program to extract ASCII as 7-bit or binary data as nibbles from the MIDI SysEx dumps from the amidi tool:

#include <stdio.h> #include <string.h> int main(int argc, char **argv) { if (argc != 2 || (strcmp("ascii", argv[1]) && strcmp("nibbles", argv[1]))) { fprintf(stderr, "usage: %s (ascii|nibbles)\n", argv[0]); return 1; } int ascii = ! strcmp("ascii", argv[1]); if (0xF0 == fgetc(stdin)) { if (0x7D == fgetc(stdin)) { if (ascii) { int c; while (EOF != (c = fgetc(stdin))) { if (c == 0xF7) break; fputc(c, stdout); } } else { int hi; while (EOF != (hi = fgetc(stdin))) { if (hi == 0xF7) break; int lo; if (EOF != (lo = fgetc(stdin))) { if (lo == 0xF7) break; fputc((hi << 4) | lo, stdout); } } } } } return 0; }

Eventually I will neaten all of this up and make the Amiga code have a GUI for selecting files and transfer modes, but these programs are short enough to type in by hand when no other data transfer method is available. Another feature might be to specify the filename in the protocol, so that you don't have to keep track, as well as chunked transfers with checksums for data integrity. So far these tools are a bit inconvenient, but I successfully transferred a 6kB text file from Linux to Amiga, and a 42kB OctaMED module (binary) from Amiga to Linux. My most immediate next step is transferring an ADF (disk imager) tool and a data compression tool (to speed up transfers) from Linux to Amiga and making backups of my Blitz Basic 2 disks to transfer to Linux for use in the FS-UAE Amiga emulation software.

]]>My piece RDEX will be exhibited in Folkestone in the coming days, as part of a show curated by Laura Netz in a curatorial residency at HOP Projects CT20:

Sounding DIY III. Vitalist Materialism.

Friday 06 - Sunday 15 December 2019

HOP Projects CT20

Folkestone, England, United Kingdom

www.hopprojects.orgOpening, Friday 06 December 2019, 6-9 pm

The concert, Saturday 07 December 2019 6-9 pm

Curator’s Talk, Sunday 08 December 2019, 6-9 pm

Closing party, Saturday 14 December 2019, 6-9 pm

Artists in exhibition:

Stephen Cornford (print)

Claude Heiland-Allen (projection)

BJ Nilsen (sound installation)

Greg Orrom Swan (video)

Erin Sexton (video)

xname (sculpture)

Artists at performance space:

Mother Disorder

Lou Barnell

Laura Netz

For more information, pictures, curatorial text, etc check the links above.

]]>
Melinda Green's webpage
The 4D Mandel/Juli/Buddhabrot Hologram
has a nice video at the bottom, titled
*ZrZi to ZrCr - only points Inside the m-set*.
I recalled my 2013 blog post about the
Ultimate Anti-Buddhabrot
where I used Newton's method to find the limit Z cycle of each C value
inside the Mandelbrot set and plotted them. The (anti-)Buddhagram is
just like the (anti-)Buddhabrot, but the Z points are plotted in 4D space
augmented with their C values. Then the 4D object can be rotated in
various ways before projection down to 2D screen, possibly via a 3D step.

My first attempt was based on my ultimate anti-Buddhabrot code, computing all the points in a fine grid over the C plane. I collected all the points in a large array, then transformed (4D rotation, perspective projection to 3D, perspective projection to 2D) them to 2D and accumulated with additive blending to give an image. This worked well for videos at moderate image resolutions, achieving around 6 frames per second (after porting the point cloud rasterization to OpenGL) at the highest grid density I could fit into RAM, but at larger sizes the grid of dots became visible in areas where the z→z²+c transformation magnified it.

Then I had a flash of inspiration while trying to find the surface normals for lighting. Looking at the formulas on Wikipedia I realized that each "pringle" is an implicit surface \(F_p(c, z) = 0\), with \(F_p(c, z) = f_c^p(z) - z\) and the usual \(f_c(z) = z^2 + c\). \(p\) is the period of the hyperbolic component. Rendering implicit surfaces can be done via sphere-marching through signed distance fields, so I tried to construct a distance estimate. I tried using \(|F_p(c, z)| - t\) as a first try, where \(t\) is a small thickness to make the shapes solid, but that extended beyond the edges of each pringle and looked very wrong. The interior of the pringle has \(\left|\frac{\partial F_p}{\partial z}\right| \le 0\) so I added that to the distance estimate (using max() for intersection) to give:

float DE(vec2 c, vec2 z0) { vec2 z = z0; vec2 dz = vec2(1.0, 0.0); float de = 1.0 / 0.0; for (int p = 1; p <= MaxPeriod; ++p) { dz = 2.0 * cMul(dz, z); z = cSqr(z) + c; de = min(de, max(length(z - z0), length(dz) - 1.0)); } return 0.25 * de - Thickness; }

Note that this has complexity linear in MaxPeriod, my first attempt was quadratic which was way too slow for comfort when MaxPeriod got bigger than about 10. The 0.25 at the end is empirically chosen to avoid graphical glitches.

I have not yet implemented a 4D raytracer in FragM, though it's on my todo list. It's quite straightforward, most of the maths is the same as the 3D case when expressed in vectors, but the cross-product has 3 inputs instead of 2. Check S. R. Hollasch's 1991 masters thesis Four-Space Visualization of 4D Objects for details. Instead I rendered 3D slices (with 4th dimension constant) with 3D lighting, animating the slice coordinate over time, and eventually accumulating all the 3D slices into one image to create a holographic feel similar to Melinda Green's original concept.

Source code is in my fractal-bits repository:

]]>git clone https://code.mathr.co.uk/fractal-bits.git

My generative techno patch Dynamo is being exhibited at Galerije VN, Zagreb, Croatia, 12 November to 7 December 2019, as part of the group show free_art_-_source/2019 organized by Format C. Opening hours Mon-Fri 8-20:00, Sat 8-14:00.

On Saturday 23rd November, also in Zagreb, I'll be leading a free workshop on live-coding audio in the C programming language using Clive. Note that clive does not work on Windows or Mac.

clive is a C audio live-coding skeleton. It allows you to hot-swap DSP processing callbacks, providing not much more than automatic C code recompilation, object code reloading, and state preservation.

The workshop will work on dependencies installation, basics of sysadmining and git, CLI, define DSP and examples similar to clive, sound synthesis and feedback, shaping sounds and a live performance preparation.

The workshop is conducted in English language.

To participate, you need to bring your own (GNU/Linux OS) laptop, and (optional) headphones.

Applications (open until Nov 20th 23:59CET):`hello at formatc.hr`

A bit nervous, but have been preparing intensively so it should be fine.

]]>A recent toot inspired me:

Computermusiker@hecanjog@social.radio.afAre there any sox-like tools that draw waveforms or spectrographs directly into the console's framebuffer?

It would be a really handy thing to quickly look over audio files on a remote server.

I wasn't happy with my gnuplot hack, so I created **harry**, a text-mode audio viewer:

It loads an audio file into memory, with 4 subchannels per input channel containing mean, min, max, and rms. Then it generates a mipmap chain, reducing the length progressively by factors of two. This reduction enables fast high quality zooming, at the cost of doubling the data size again. Total memory usage is about 32 bytes per sample per channel, which means a 6 minute CD track explodes to about 1 GB (from about 55 MB).

But! Literally just this minute I remembered that at the base (longest) level, all of mean, min, max, rms are identical, so I can reduce the total memory to 5/8 of the current bloat, at the cost of more complicated logic when cleaning up the buffers (double-free is not allowed). I may implement that later tonight.

When plotting a waveform the naive way, one might simply plot all the points. But audio files have many 1000s of points, and unless you are zoomed all the way in it's just too much data to deal with. One way of reducing might be to sample the data at a subset of points, but that leads to aliasing (OctaMED SoundStudio's sample editor does this, it is very bad and very confusing because the visual appearance of the waveform has little resemblance to what is really happening). The way to anti-alias is to filter, and I'm using a simple box filter kernel (neighbouring pairs of samples, non-overlapping) in the reduction stages.

So the variables that are plotted:

- mean
- This is a simple low pass filter, that approximates what a band-limited version of the signal would be. As well as for plotting, these buffers are used for variable (higher) speed audio playback with cubic interpolation, then linear interpolation between neighbouring mipmap levels.
- rms
- Raw RMS is not so useful, but it can be cooked to standard deviation
via
`sqrt(rms * rms - mean * mean)`

, and that gives a meaningful bound on the spread of the signal values in each cell - min, max
- Peaks are useful to monitor because exceeding the value range (typically -1 .. 1) leads to ugly clipping distorion.
- zero
- Silence is the baseline for everything.

At high zooms the actual waveform is visible:

file 0:00:05:53.851 mathr_-_lockaccum.wav view 0:00:00:00.058 gain 1 | speed 1 at 0:00:00:00.319 | @@@ | ^~v~@ @~~~@@@ | @@~~@@^ ^~~ ^ @@@@^@ @v v~ ~|@v @@v~@^ ^^@@@@@@@~ v v~^ @ @^ ^@@| ~~ ~@vvvvv @@^ ------@--------@----------@-------------|--------@-----------@@----------@@----- @ ~ ~ @@ | @v^^ ^@@~v ~@~^ @ v@ @v ~~ | @@@ v@@@v v@@^^ ~@@v@@ @~v~ @@@ | ~@@v @@ v@ @@@@ | v | ~@~ | ^ ^ ^@@ @ ^@@~@@@@ | @@@@@@ ~@ @^ ^ @vv@~~ ~@v ~~ ~|@v~ v @ ^^@~@~@@@@^ v@^ @ @^ @@| ~@ ^@@v v vvv@@ ------@--------@----------@----------@--|---------@-----------------------@----- ~ ~~ ~@ ^~ | ~@ ~@@ ~@~ @~ @@ ~~ @@ | v~@@~@@@@@ v v@@@^ @@@@@~ @@ @v@@v | ~@v ~ v~@ vv v@~@~ | >]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

At medium zooms the mean and standard deviation start to give a thick core while the outer min/max fringes show the peaks:

file 0:00:05:53.860 mathr_-_lockaccum.wav view 0:00:00:00.928 gain 1 | speed 1 at 0:00:00:10.303 | ^ ^ ^^^^ ^^^ ~ ^^^^ ^ ^ | +^ ^^ ^+ ^ ++~+ ~+~ ^* ~ +~~+^ ~+^+^ | ^ ^ ~+ +~ ^~~^~ ~ ~ ~~*~~*~*~+@ * @ @ +**~~ ^~^*~~++^ | ~ ~^ ^^ ^~ ^^^~ *~~~*^+**+@ * * ~^^**@****@*~~^@ *^~^ ~@@*@^+*~@*@~~~~^|^@^@+~~~~~^@~~~*^^@~~@^****@~+*@+*^@^@ @@@@@-@@@---------@---@---@--@@@-@-@-@@@|----@@@@@@@-@@-@---@@--@@@@-@-@-----@-- ~v~**~*~*@~@*v*v~ ~ @@~@*~*~@+**v~+~@~~~|@v@v*~+ ~~~ ~~@+@@v*~ ~******@*~*+@v~v~ ~~+~+~*+~@ @ @ ~vv~~v*v*+*~ v+v~ +v|~ ~ ~vv v v vvvvv~ ~+ @~~~~~~+~+@v* v @ vv++v+*vv+ ~ ~ v v ~ ~+~ v v v | v vv v+v++vvv+v~ ~ ~ vv v~ v v v vvv | v vv v v v ^ | ^ ^^ ^^^ ^ ^^+ ^ | ^^ ^ ^^^ ^ ~^+~ ~+~ ~ ^ ^ ~ +~~^^ ~^^^+^^ | ^ ^ ^^ ~+ ~~ ++~^~ ^ ~ ^ *~~*^*+*~^@ ~ ~ * +**~~ *~+~~~~^^ | ~ ~ ^^^ ^ ++^ ~^~~ *~~**^+~*+@ ~ * ~^^**@*~*~@*+~^@ @^@ ^ ~@@*@^**+@*@*~+^~| @^@~^^@+~^~^^~@~ @+*@^@***@~~*@+*^@^@ @@@@@-@@@--@------@----@--@--@---@-@@--@|----@@--@@@@@@---@--@---@-@-@-@-------- ~v~*~~*~*@~**v*v~ v @@@*~~**@*@@+~~+~@@~|@v@v*~@ ~~* ~~@~@~ @~ @*~@**~@*~~v@+@+~ ~++~v**+~@ @ * ~v ~++*~*~~vvv+++ +v|v ~ ~vv v ~ v~ ~ ~+ ~~+**~v*~v@ *v~v@ +v++ ~*vv+ ~ @ v vvv~ *++ vv+ v | v +v~~v ~v * ~ v ~ v vv v~ v v v ~vv v | v v v ~ [[>]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]

At low zooms the long-time-scale dynamics are visible:

file 0:00:01:30.650 01.Yawning_Zeitgeist_Intro_(Freestyle).flac view 0:00:00:59.443 gain 1 | speed 1 at 0:00:00:44.582 ^ | ^ ^ ^ ^ + ^^| ^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^+^^^^ ^^ +^+ ^ ^+^+^++| ^^ ^^ ^++ ^ ++^^^+ ^ ^+ ^ ^ ^ ^^ ^^^ + +^+^+^^+^ ^++++++^++^+++^+^+++++++|^++^ ^^++^ +++^^^+^++++++^+^++^+^+^+^++ +~~~+^~~~~~~~+~~~~+++~+~~~++~++~+~+~+~~~|++~~^+++~+^~~~++~+~~~~+~+++~~+~++++~+~~ @@@-@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ ++v@~~+~~~~+~~+v~+~~++++~~~~~++~~~++~~~+|~~++~~~~~v~+~~+~+~v~~++++~~+~+~~~+~~+~~ +v v+v+vvv++v+v vv+v+++vv+++v+++v+++++++|v+++ vv+v +++++++ ++v+++++v++++v++vv++ v v v vv v v vvv vvv +vv vv++vv+| vvv v vv+vvvv vv +v+vv vvvv vv +v v v+ v| v v v v ^ v | +^ ^ ^ ^ ^ ^^^ | ^ ^^ ^ ^^^^^ ^ ^ ^ ^^ ^ ^ ^++^^^ ^+^+ +^^^^+^+++^| +^ ^ ^ ^++^ ^+ ^+++++^^ ^+^+^ ^^ + ++^^+ ^^^^^^^+^ ^++++++^++++^+++++++++++|^++^ +^^+ ++++^++^++++++++^+++++^++^^+ ~+~~~^~+++++~+~^~~~~+~~~~~++++~~+~~~~~~~|~+++^+~~+~^+~~~~~~~+~~+~~~~+~~~~~~~~+++ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ +++~+++~++~+~++v~~~~~+~~~~~~~~++~~~+~+~+|v+++~++~+v~~~~~+~~~~+~~~~~++~~~~~~~~+~~ ++v+vv+vvv++vvv +++v+++v++++++++++++++++| +++ +v++ v++++++v++++++++v++++++++v++ +v v v vv v++ vvv v+v+v++vvvvv+vv+| v+v v vv ++vvvv vvv+v++v vvv+vvvv v+ v vv v v vv v v| v vv v vv v v [[[[[[[[[[[[[==========================>==========================]]]]]]]]]]]]]]

These ASCII grabs don't do it justice, because the real code uses ANSI control sequences via ncurses to make parts more or less bold. It also optionally plays audio using SDL2 (which has a PulseAudio backend that can work remotely over local network) with a synchronized play head (in two modes), with whole file or A-B subregion looping (same key controls as MPV).

Another cool thing in the build system embeds the sources into the executable using the linker, so it's easy to fulfil your AGPLv3+ license obligations. A command line flag writes out the files again. Check the Makefile for details, and the README (duplicated to the harry website via Pandoc and some hand editing) for the documentation.

]]>In the last months, after adding EXR support to KF, I worked on some tools to help process the output data. Most imagery software only expects RGB(A) or maybe YUV images, and even so I had to submit a patch to DarkTable to get it to ignore any extra channels. My tools have a website here:

So far there are three tools, a simple channel extractor (which needs more work to be truly useful), a preview extractor (which is fine for what it does), and a tile assembler. The tile assembler has two modes, one for side by side images (as is normal), and a "stratified" mode that interleaves pixels of slightly-offset images (as rendered by KF in one of its tiling modes). I had a nightmare debugging that because I forgot that "enable guessing" in KF combined with "log de" colouring is not a sufficiently accurate approximation for smooth colours - turned off guessing and the grid-like subtile artifacts disappeared.

I had to add channel filtering to the pipeline, because an EXR file at 64000x36000 with all the raw channels from KF would be over 50GB, much more RAM than I have. I added the filter to exrtactile because it was easier, but I should add a filter to KF too, to avoid saving data that isn't wanted. Even so, the image dimensions were too large for most of the software I tried, but downscaling to 16000x9000 with ImageMagick let me load it in DarkTable. I had to free up some space for the pixel cache of ImageMagick too, the first attempt failed with disk full.

]]>Next week:

Tatsuru Arai / Bioni Samp / mathr&netzSaturday, 16 November 2019 from 20:00-22:30

IKLECTIK, Old Paradise Yard, 20 Carlisle Lane, SE1 7LG London, United Kingdom

We'll be playing with hardware synthesizers, accompanied by psychedelic fly vision projection:

Some text about our performance:

The fly vision algorithm is based obviously on the eyes of flies,

"A compound eye is a visual organ found in arthropods such as insects and crustaceans. It may consist of thousands of ommatidia, which are tiny independent photoreception units that consist of a cornea, lens, and photoreceptor cells which distinguish brightness and color. The image perceived by the arthropod is a combination of inputs from the numerous ommatidia, which are oriented to point in slightly different directions. Compared with single-aperture eyes, compound eyes have poor image resolution; however, they possess a very large view angle and the ability to detect fast movement." (Wikipedia)The visualization code takes an image from a single lens webcam, and processes it to emulate the ommatidia. Arranged in a hexagonal grid, which is an optimal packing reached by evolution, each cell has a wide angle, with overlapping edges repeating parts of the image and enhancing the drama of moving objects. The colour of the cells are modified according to the volume levels of three frequency bands of audio (bass, mids, treble). Over time the size of the cells are modulated, zooming disconcertingly.

Looking forward!

]]>Quoth Wikipedia:

In mathematics, the Haar wavelet is a sequence of rescaled "square-shaped" functions which together form a wavelet family or basis. Wavelet analysis is similar to Fourier analysis in that it allows a target function over an interval to be represented in terms of an orthonormal basis. The Haar sequence is now recognised as the first known wavelet basis and extensively used as a teaching example.

The principle advantage of wavelet transform vs discrete Fourier transform is a different frequency/time localisation tradeoff: wavelets give better localisation in time for higher frequencies and better localisation in frequency for lower frequencies. This matches up better with human perception too.

The main advantage of 1D Haar wavelets in particular is that they are very easy to implement: given the source data block (power of two size), compute the sum and difference of each (non-overlapping) pair of consecutive samples. Put the sums in the first half of the array and the differences in the second half of the array, then repeat the process on the half containing the sums until there is only one sample. This process is reversible: no data is lost. I use a different normalization: dividing by two, so that the levels don't get higher in lower octaves. The DC bin then contains the mean instead of the sum. Source code in C:

// BLOCKSIZE must be a power of two float array[2][BLOCKSIZE]; // compute Haar wavelet transform // input/output in array[0] using temporary array[1] for (int length = BLOCKSIZE >> 1; length > 0; length >>= 1) { for (int i = 0; i < length; ++i) { float a = array[2 * i + 0]; float b = array[2 * i + 1]; float s = (a + b) / 2; float d = (a - b) / 2; array[1][ i] = s; array[1][length + i] = d; } for (int i = 0; i < BLOCKSIZE; ++i) { array[0][i] = array[1][i]; } } // compute inverse Haar wavelet transform // input/output in array[0] using temporary array[1] for (int length = 1; length <= BLOCKSIZE >> 1; length <<= 1) { for (int i = 0; i < BLOCKSIZE; ++i) { array[1][i] = array[0][i]; } for (int i = 0; i < length; ++i) { float s = array[1][ i]; float d = array[1][length + i]; float a = s + d; float b = s - d; array[0][2 * i + 0] = a; array[0][2 * i + 1] = b; } }

A classic FFT algorithm for audio is the "timbre stamp", applying the spectrum of one signal to another. A similar thing can be done with wavelets: compute the Haar wavelet transform of both signals, then compute the RMS energy per octave of the control signal. Multiply each octave of the carrier signal by the corresponding energy factor, then do compute the inverse Haar wavelet transform. As with FFT algorithms, windowed overlap-add resynthesis gives a smoother sound. Example audio snippet:

speech

The energy per octave representation is quite compact (for audio as humans can hear it it only makes sense to compute around 10 octaves), but even so it can give quite a lot of information about a signal. But most interesting signals are not stationary: the energy per octave changes over time. One simple statistical metric that can be used to quantify amount of change is the standard deviation (which is the square root of variance: variance is the expected value of the squared differences from the mean) of each octave bin.

Variance captures deviations from a mean value, but it doesn't capture
much about how they vary. One idea I implemented was to compute *another*
set of Haar wavelet transforms, this time capturing how the energy in each
octave changes over time. Eventually I will subtract the mean from each
energy vector before windowing, to avoid modulating the DC bin, but for
now I just used a rectangular window. I then calculated the transform
of each octave separately, then computed the RMS energy of each octave of
"rhythm". This gives a 2D representation of energy vs octave (audio) vs
octave (rhythm). Comparing various source audio files:

The result seems like it might work reasonably well as an audio fingerprint. One next step in my research will be trying that out, to see if I can identify different audio files from a short snippet.

I tried reversing the process, going from the rhythm/octave fingerprint
to audio, by stamping it on white noise. But it didn't work very well:
the white noise had no energy in its rhythms (apart from DC), so amplifying
zero still gives zero. I tried a feedback process but that amplified the
DC-rhythm energy-per-octave spectrum over and over, so the result was very
Alvin Lucier *I am sitting in a room*. A different approach that
showed promise was normalizing the control rhythm/octave print first,
and a variation involving normalizing the rhythm/octave print of the
carrier is something I want to try next. Example audio snippets:

white

pink

brown

speech

music

One problem with the energy per octave (rhythm) representation is that it doesn't capture interactions between different octaves over time. One way of modelling sequential relationships is a Markov chain: it typically operates on discrete symbols at discrete time steps, and the probability of the next symbol generated depends on the recent history of generated symbols. Training a Markov chain is quite simple, but converting energy per octave vectors to symbols is not so easy. I ended up training a 2D self-organizing map to reduce the 11D vectors to a small 2D grid (8x8), and the index into the grid becomes the symbol for a first-order Markov chain. Then I generate symbols with the Markov chain and timbre stamp noise with the corresponding node vectors from the self-organizing map. For bonus points I could in future try Earth-mover's distance instead of regular Euclidean distance for the SOM metric. Example audio snippets:

white

pink

brown

speech

music

References for source audio (all 44100Hz stereo 16bit):

- white
- White noise generated with Audacity, 10mins
- pink
- Pink noise generated with Audacity, 10mins
- brown
- Brownian noise generated with Audacity, 10mins
- speech
- BBC Radio 4 - The Archers Omnibus 27/10/2019 76mins
- music
- CPI - Every Stone Held (2019) 95mins

A website for this project, including links to source code, is at mathr.co.uk/disco.

]]>Computerized representations of numbers come in a few different flavours.
The most common is an `int`

, which is typically 4 bytes long, or
32 bits. A bit is either 0 or 1, and a byte is the smallest unit addressable
by the CPU and consists of 8 bits, able to store 2^{8} or 256 distinct
values. An `unsigned int`

stores numbers from `0`

to
`2`

, and a ^{32}-1`signed int`

uses
twos-complement to represent negative numbers: the most significant bit is
`1`

for negative numbers, and *sign extension* means it
really represents an infinite stream of `1`

s extending to the left.
You can find the twos-complement of a number by inverting all the bits and then
add 1 to the whole number.

Computers also have `float`

numbers, which have a floating point,
like scientific notation for numbers. The number of significant digits is fixed,
but you can represent very large numbers and very small numbers by combining the
mantissa digits with an exponent, which moves the point separating the fractional
part. I'll talk more about this in a future post. Floating point typically uses
a magnitude-and-sign representation instead of twos-complement.

Fixed-point numbers are somewhere in between, they can represent numbers with fractional parts, but the number of digits after the point is fixed, so very small numbers cannot be represented as accurately as numbers with magnitude near 1. For some purposes this is just fine, as the numbers you are concerned with always have a magnitude near 1. For example, when calculating the Mandelbrot set, each pixel's c value is between -2-2i and +2+2i, and the interesting parts have |c|>0.25 too. The escape time algorithm iterates z→z²+c starting from 0 until |z|>2 (or you give up after an arbitrary iteration count limit) which means most the z values have magnitude near 1. This makes high precision fixed-point a good fit: you need more digits when zooming in, otherwise you can't represent neighbouring pixels' c values, but you don't need the more-complex algorithms of floating-point numerics because you don't need to represent very small or very large numbers.

So how do you implement fixed-point numerics on a computer? I used
`unsigned int`

limbs, which are essentially digits in base 2^{32},
and stored numbers as an array of limbs and two numbers counting the number of
limbs before the point and the number of limbs after the point. My code is quite
general, with mixed precision operations, but this aspect is not so well tested as
the Mandelbrot set calculations only need 1 limb before the point. The most primitive
operations are reading a limb from the vector, which does sign extension in
twos-complement for limbs larger than the stored array, and pads with 0 for smaller
limbs; and writing a limb to a vector, which has some assertions to crash fast
instead of corrupting memory with an out of bounds write.

**-a** is implemented by complementing all the limbs and adding 1
in the least significant place, propagating carries along towards the more significant
limbs. Carries can be calculated by comparing the result of addition with the
addends, if the result is smaller then it overflowed (`unsigned int`

arithmetic is modulo 2^{32}) and you need to add 1 to the next limb.

**a<<n** and **a>>n** can be calculated
by reading two adjacent limbs into a double-wide `unsigned long long int`

(there is disagreement over whether `long`

is 32bit or 64bit on
different systems, `long long`

is 64bit almost everywhere I think) and
shift the correct 32bit part out of it while looping over the arrays, being careful
not to clobber the array you are reading from in case it is aliased to the array
you are writing to (it is nice to be able to shift in-place to save memory).

**a+b** is implemented by zipping from least significant limb to
most significant, propagating carries along and adding them to the next pair of
limbs. **a-b** is implemented similarly, only propagating borrows
to subtract from the next pair. Borrow is calculated by the subtracted limb being
larger than the limb from which it is subtracted. OpenGL 4 / GLSL 400 and above
have `uaddCarry()`

and `usubBorrow()`

functions which make
this easier to implement, but I wrote my implementation in OpenCL / C / C++ which
afaik doesn't have these.

**a ^{2}** is implemented as an optimization of multiplication
taking advantage of symmetry to optimize duplicated work like

`x*y+y*x`

as `(x*y)<<1`

which may make a small difference. Multiplication
`a`

by every limb in `b`

and collect up the results, so that is what I did as
a first pass for simplicity (more efficient algorithms like Karatsuba multiplication
may be interesting to investigate later). It differs from integer multiplication as
found in libgmp (for example) because the least significant limbs of the result can
often be discarded or not computed, at least if we don't care about correct rounding
(all the operations I implemented simply truncate towards negative infinity).
I made a diagram (click for bigger) showing how I implemented it (actually a version
of this diagram came before I started writing any code at all):The input numbers are fed in from the right hand side and the bottom, into limb-by-limb multiplication (32×32→64bit) which outputs the most significant and least significant limbs of the result to two wide accumulator busses that flow diagonally down and left. At the end the accumulators are combined into limbs by propagating carries upwards. The accumulators don't need to be super wide, I think they need O(log(number of limbs)) extra bits beyond the width of the limb, but 64bit is certainly plenty and is usually available.

One final thing to note, I tried benchmarking both big-endian and little-endian limb orderings, to see how that affected things. To my surprise big-endian came out a little faster, surprising because most operations operate over limbs from the little end first, but I think that was because I was only testing with 1 limb before the point, which potentially made an addition (statically known to be +0) be completely omitted by the optimizer. At zoom 1e90, which corresponds to about 10 limbs per number, simple escape time fixed-point computation of a Mandelbrot set location on my AMD RX 580 GPU was only about 12-13 times slower (total elapsed wall-clock time) than KF using advanced perturbation and series approximation algorithms on my AMD Ryzen 7 2700X CPU. I want to find a way to compare total power consumption, maybe I need to get a metering hardware plug or some such.

You can find my implementation here:
`git clone https://code.mathr.co.uk/fixed-point.git`