Neural Network Cellular Automata

2021-09-06

Okay. A random post on the /r/cellular_automata subreddit inspired me.

Let’s generate a cellular automata where each pixel updates based on a neural network given as input:

  • The x/y coordinates (scaled to the range 0-1)
  • An optional random value (to make it more dynamic)
  • A variety of neighboring data, such as:
    • The number of neighbors that are ‘active’ (> 50% white), ranges 0-8 scaled to 0-1. This should allow Conway's Game of Life
    • The RGB values of all neighbors (allows a superset of the above)
    • Gradients, subtract color value of the left from the right so that you get edges and side to side movement

Let’s do it!

First off, I’m going to use brain.js for the neural networks. It’s quick and easy to use with one big exception: I couldn’t figure out how to generate random weights. It’s not really something that neural networks are ‘designed’ to do in general and I did eventually figure it out, but it was a bit of a pain.

Things to play with:

  • The regenerate button will generate a new random network
  • The debug button will toggle debug information under your mouse
  • The network button will toggle a JSON view of the current neural network
  • The svg button will toggle a rendered SVG of the current neural network
  • The import button can take a JSON object containing a network (copied from the network tab) and load it back in
  • gridSize controls how large pixels are (making them too small will cause performance issues)
  • showBorder will render pixel borders
  • outputAddNoise will add noise to the pixels after they are generated by the network
  • inputAddNoise will add noise to the first input to the neural network (otherwise it will be zero)
  • inputAddXY will add X/Y as the second and third input to the neural network (otherwise zero)
  • biasScale and weightScale control the generated values of the neural network; the given values generally have output, too small or large will mean all values end up the same
  • activation changes the neural network activation function
  • colorInputMode controls how additional inputs are added:
    • neighbor count is counting how many neighbors each pixel has, ranges 0-8 scaled to 0-1
    • neighbor RGB passes the R, G, B value of the 8 neighboring pixels in as 0-1 each (for 24 input channels)
    • gradiant x/y will pass the difference between right/left and top/bottom, making patterns that tend to move
    • none will ignore neighbor hood, using only noise/x/y (if on above)
  • hiddenLayers should be a JSON list of number defining how many neurons are in each hidden layer of the neural network; more is more complicated behavior but slower

Below the code is broken up a bit, we have:

  • init.js initializes the sketch, not super interesting
  • input-modes.js defines the different input modes above, basically it stores how many values will be generated and a function to generate them
  • input.js takes those values and adds noise/x/y if requested
  • reset.js resets the sketch, this also does the work of setting up the random neural network, which is one of the most interesting bits of the code
  • draw.js updates the network (using a double buffer so that we don’t modify the input values as we’re generating them) and draws them to the screen

Let’s check it out!


let gui;
let params = {
  gridSize: 10,
  gridSizeMin: 1,
  gridSizeMax: 20,
  showBorder: true,
  outputAddNoise: false,
  inputAddNoise: true,
  inputXY: true,
  biasScale: 2,
  biasScaleMin: 1,
  biasScaleMax: 10,
  biasScaleStep: 0.1,
  weightScale: 7,
  weightScaleMin: 1,
  weightScaleMax: 10,
  weightScaleStep: 0.1,
  activation: ['sigmoid', 'relu', 'leaky-relu', 'tanh'],
  colorInputMode: undefined,
  hiddenLayers: '[10, 10, 10, 10]',
};

let data = []
let gridWidth = 0
let gridHeight = 0

let outputSize = 3
let inputSize = 3 + (9 * outputSize) // x, y, and rgb of 3x3 area
let hiddenLayers = [10, 10, 10, 10]

let net = undefined

function toggleVisible(el) {
  if (el.style.display === 'none') {
    el.style.display = 'block'
  } else {
    el.style.display = 'none'
  }
}

function makeToggle(name, type) {
  let el = document.createElement(type)
  el.id = name
  el.style.display = 'none'
  
  let button = document.createElement('button')
  button.innerText = name
  button.addEventListener('click', function(e) { e.preventDefault(); toggleVisible(el); return false; })
  button.href = '#'

  return [button, el]
}

function setup() {
  createCanvas(400, 400);

  let button = document.createElement('button')
  button.innerText = `regenerate`
  button.addEventListener('click', function(e) { e.preventDefault(); reset(); return false; })
  button.href = '#'
  document.body.appendChild(button)
  
  gui = createGuiPanel();
  gui.addObject(params);
  gui.setPosition(420, 0);

  let [debugButton, debugEl] = makeToggle('debug', 'pre')
  let [networkButton, networkEl] = makeToggle('network', 'pre')
  let [svgButton, svgEl] = makeToggle('svg', 'div')

  for (let el of [debugButton, networkButton, svgButton, debugEl, networkEl, svgEl]) {
    document.body.appendChild(el)
  }

  let importButton = document.createElement('button')
  importButton.innerText = `import`
  importButton.addEventListener('click', function(e) { 
    e.preventDefault();
    js = JSON.parse(prompt('Neural network JSON:'))
    reset(js)
    return false; 
  })
  importButton.href = '#'
  document.body.appendChild(importButton)

  document.body.appendChild(document.createElement('br'))

  reset()
}


let inputModes = {
  'neighbor count': {
    size: 1,
    inputAt: (x, y) => {
      let neighbors = 0
      for (let xd = -1; xd <= 1; xd++) {
        for (let yd = -1; yd <= 1; yd++) {
          if (xd === 0 && yd === 0) 
            continue
          
          let g = (getData(x, y, 0) + getData(x, y, 1) + getData(x, y, 2)) / 3
          if (g > 0.5)
            neighbors += 1
        }
      }
      return [neighbors / 8.0]
    }
  },
  'neighbor RGB': {
    size: 24,
    inputAt: (x, y) => {
      let values = []
      for (let xd = -1; xd <= 1; xd++) {
        for (let yd = -1; yd <= 1; yd++) {
          if (xd === 0 && yd === 0) 
            continue
            
          for (let c = 0; c < outputSize; c++) {
            values.push(getData(x, y, c)) // RGB
          }
        }
      }
      return values
    }
  },
  'gradiant x/y': {
    size: 2,
    inputAt: (x, y) => {
      let gl = getData(x - 1, y, 0) + getData(x - 1, y, 1) + getData(x - 1, y, 2)
      let gr = getData(x + 1, y, 0) + getData(x + 1, y, 1) + getData(x + 1, y, 2)
      let gu = getData(x, y - 1, 0) + getData(x, y - 1, 1) + getData(x, y - 1, 2)
      let gd = getData(x, y + 1, 0) + getData(x, y + 1, 1) + getData(x, y + 2, 2)
      
      return [
        gr - gl,
        gd - gu
      ]
    }
  },
  'none': {
    size: 0,
    inputAt: (x, y) => [],
  }
}

params.colorInputMode = Object.keys(inputModes)


function inputAt(x, y) {
  let input = [
    params.inputNoise ? Math.random() : 0,
    params.inputXY ? 1.0 * x / gridWidth : 0,
    params.inputXY ? 1.0 * y / gridHeight : 0,
  ]
  input = input.concat(inputModes[params.colorInputMode].inputAt(x, y))
  return input
}


function reset(network) {
  inputSize = 3 + inputModes[params.colorInputMode].size
  hiddenLayers = JSON.parse(params.hiddenLayers)
  
  net = new brain.NeuralNetwork({
    hiddenLayers,
    activation: Array.isArray(params.activation) ? params.activation[0] : params.activation,
  })
  
  gridWidth = width / params.gridSize
  gridHeight = height / params.gridSize
  
  // Train it on random data to initialize
  let trainingData = []
  for (let i = 0; i < 1; i++) {
    let input = []
    for (let j = 0; j < inputSize; j++) {
      input.push(Math.random())
    }
    
    let output = []
    for (let j = 0; j < outputSize; j++) {
      output.push(Math.random())
    }
    
    trainingData.push({input, output})
  }
  net.train(trainingData, { iterations: 1 })
  
  // Rewrite the weights
  if (network) {
    net = net.fromJSON(network)
  } else {
    let js = net.toJSON()
    for (let layer = 1; layer < js.layers.length; layer++) {
      for (let neuron in js.layers[layer]) {
        let newBias = params.biasScale * Math.random() * 2 - params.biasScale
        js.layers[layer][neuron].bias = newBias
        
        for (let weight in js.layers[layer][neuron].weights) {
          let newWeight = params.weightScale * Math.random() * 2 - params.weightScale
          js.layers[layer][neuron].weights[weight] = newWeight
          
        }
      }
    }
    
    net = net.fromJSON(js)
  }
  
  data = []
  for (let x = 0; x < gridWidth; x++) {
    var row = []
    for (let y = 0; y < gridHeight; y++) {
      var pixel = []
      for (let c = 0; c < outputSize; c++) {
        pixel.push(Math.random())
      }
      row.push(pixel)
    }
    data.push(row)
  }
  
  document.getElementById('network').innerText = JSON.stringify(net.toJSON(), null, 2)
  params.network = JSON.stringify(net.toJSON())
}


function getData(x, y, c) {
  try {
    return data[x][y][c] || 0
  } catch(ex) {
    return 0
  }
}

function draw() {
  if (params.showBorder) {
    stroke(0)
  } else {
    noStroke()
  }

  let buffer = []
  for (let x = 0; x < gridWidth; x++) {
    var row = []
    for (let y = 0; y < gridHeight; y++) {  
      let pixel = net.run(inputAt(x, y))   
      
      if (params.addNoise) {
        pixel[0] += (Math.random() - 0.5) / 100
        pixel[1] += (Math.random() - 0.5) / 100
        pixel[2] += (Math.random() - 0.5) / 100        
      }
      
      row.push(pixel)
    }
    buffer.push(row)
  } 
  data = buffer
  
  for (let x = 0; x < gridWidth; x++) {
    for (let y = 0; y < gridHeight; y++) {
      fill(255 * getData(x, y, 0), 255 * getData(x, y, 1), 255 * getData(x, y, 2))
      rect(x * params.gridSize, y * params.gridSize, params.gridSize, params.gridSize)
    }
  }
  
  let gx = Math.round(mouseX / params.gridSize)
  let gy = Math.round(mouseY / params.gridSize)
  let r = Math.round(255 * getData(gx, gy, 0))
  let g = Math.round(255 * getData(gx, gy, 1))
  let b = Math.round(255 * getData(gx, gy, 2))
  
  let debugData = {
    frame: frameCount,
    pt: {gx, gy},
    rgb: {r, g, b},
    output: net.run(inputAt(gx, gy)),
    input: inputAt(gx, gy),
  }
  
  document.getElementById('debug').innerText = JSON.stringify(debugData, null, 2)
  document.getElementById('svg').innerHTML = brain.utilities.toSVG(net)
}

Take a look at the code and feel free to ask me if anything needs more explication. Of particular interest are:

Generating a random brain.js network

net = new brain.NeuralNetwork({
  hiddenLayers,
  activation: Array.isArray(params.activation) ? params.activation[0] : params.activation,
})

// Train it on random data to initialize
let trainingData = []
for (let i = 0; i < 1; i++) {
  let input = []
  for (let j = 0; j < inputSize; j++) {
    input.push(Math.random())
  }
  
  let output = []
  for (let j = 0; j < outputSize; j++) {
    output.push(Math.random())
  }
  
  trainingData.push({input, output})
}
net.train(trainingData, { iterations: 1 })

// Rewrite the weights
let js = net.toJSON()
for (let layer = 1; layer < js.layers.length; layer++) {
  for (let neuron in js.layers[layer]) {
    let newBias = params.biasScale * Math.random() * 2 - params.biasScale
    js.layers[layer][neuron].bias = newBias
    
    for (let weight in js.layers[layer][neuron].weights) {
      let newWeight = params.weightScale * Math.random() * 2 - params.weightScale
      js.layers[layer][neuron].weights[weight] = newWeight
      
    }
  }
}
net = net.fromJSON(js)

This is a bit of a pain, but I finally got it to work. In order to have value to rewrite, you have to train it at least once. Originally I trained on random data, but that didn’t work nearly as well. So now instead, we train, export to JSON, rewrite all of the weights and biases, then load it back in. It’s messy, but it works. Perhaps I’ll make a PR for brain.js to allow random weights. Perhaps it does already and I just can’t find it. :)