# Multi-Scale Feedback Network

Stroboscopic drone web-based noise instrument.

STROBE WARNING mathr.co.uk/msfn STROBE WARNING

# 1 Interface

Video pause/play buttons control graphical processing (also stops when tab is hidden).

Audio stop/pause/play buttons control audio processing (also silenced when tab is hidden). Slider controls audio volume.

Resolution control allows choosing higher/lower resolution according to your GPU power. Resolution is automatically reduced if frame rate drops too low. Resolution can only be increased if frame rate is high enough.

Isotropic checkbox uses a higher quality downscaling algorithm when generating the multiscale pyramid. This reduces grid artifacts and axis alignment, but costs more computation and doesn’t always work on mobile (symptoms: two frames swap without changing)..

Cubic checkbox uses a higher quality interpolation algorithm when reading the multiscale pyramid. This reduces grid artifacts and axis alignment, but costs more computation.

Continuous checkbox turns off the thresholding step of the algorithm, replacing it with a tanh approximation.

! button reseeds image with noise.

?! button randomizes controls.

Randomize checkbox automatically randomizes and reseeds if statistics indicate that the image is bad.

Mixer checkbox shows/hides mixer.

Controls checkbox shows/hides controls.

Statistics checkbox shows/hides statistics.

# button saves parameters to browser URL hash, for bookmarking or sharing.

# 1.1 Mixer

Audio: amount of each of 4 layers per two audio channels (left, right). Amounts can be negative for antiphase effect.

Video: amount of each of 4 layers (plus a constant) per three video components (red, green, blue).

Mixer settings are not affected by the randomizer.

# 1.2 Controls

Each cell of the cellular automaton can -1 or +1, thresholded against 0 at the end of the process (unless continuous checkbox is activated, in which case any value between -1 and +1 is allowed and thresholding is disabled).

There are 4 layers, with one column of controls per layer.

Speed (S): how much to mix newly computed value with original value each iteration.

Noise (T): adds noise to the layer (after coupling).

Bias (h): adds a constant offset to the layer (after coupling).

Coupling (c): matrix mixer for combining layers together.

Feedback (g): controls feedback gain for each scale level (0 is fine, 11 is coarse). The up and down buttons shift all the gains, allowing for example to double or halve the size of features when changing resolution to maintain global appearance.

# 1.3 Statistics

Statistics are computed four times a second.

Mean (µ): mean per channel.

Standard deviation (σ): standard deviation per scale levels per channel.

Delta (δ): difference between successive frames.

# 2 How It Works

(To be written…)

# 3 TODO

handle loss of webgl context
motion blur
multiscale delta statistics
better heuristics for bad images
webcam input with custom 3 channel to 4 layer mixer
audio pitch control
audio input (map inter channel difference of energy per octave of sound to a layer’s multiscale feedback gains)
audio out to in loopback ability