Introduction

In this tutorial we’ll be making an iOS app that renders the Mandelbrot Set and allows us to pan and zoom around it exposing beautiful and complex imagery.

The end result will look like this:

This tutorial we’ll be based on Fractals in Xcode 6 we’ll be using the shader code from there as a starting point.

Expand below to see the shader code.

Shader code void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] vec3 color = vec3 ( 0.0 , 0.0 , 0.0 ); // initialize color to black vec2 z = position; // z.x is the real component z.y is the imaginary component // Rescale the position to the intervals [-2,1] [-1,1] z *= vec2 ( 3.0 , 2.0 ); z -= vec2 ( 2.0 , 1.0 ); vec2 c = z; float it = 0.0 ; // Keep track of what iteration we reached for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; if ( dot (z,z) > 4.0 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } if (it < float (iterations)) { color. x = sin (it / 3.0 ); color. y = cos (it / 6.0 ); color. z = cos (it / 12.0 + 3.14 / 4.0 ); } gl_FragColor = vec4 (color, 1.0 ); } [collapse]

Download the Starter Project to follow along with this tutorial. You can find the final code at the end of the article.

Project setup

The GameScene.sks file contains a single sprite named fractal that fills the full scene with the shader Fractal.fsh attached to it.

Fractal.fsh contains the shader code from above

GameViewController.swift contains code for setting up the game scene.

GameScene.swift is empty

If you run the code right now you’ll get the following result:

Notice that the aspect ratio is fixed at 3/2. We’ll have to make it adjust based on the screen size.

Also, the image is static and you can’t interact with it in any way.

Setting up the view

We’ll be using a transparent scrollview for handling panning and zooming. The scrollview will automatically keep track of our position and our zoom level in the fractal.

Open the Main.storyboard file and drag a scrollview into the view. Make the scrollview fill the view and add constraints for width, height leading space and bottom space.

Set the scrollview’s max zoom level to 100000. We’ll be able to magnify the fractal up to one hundred thousand times! We can’t go any higher than that with this approach because we’ll reach the precision limit of float .

Drag a view into the scrollview. This view will be used for zooming the scrollview. The view itself will not display anything, we’ll use the contentOffset and zoom properties of the scrollView to update our shader. Make sure that the view fills the scrollView and you set constraints for width, height leading space, trailing space, top space and bottom space. Also set the view’s background color to Clear Color.

Next we’ll hook up the outlets we need and the scrollView’s delegate.

Drag outlets for the scrollView and the scrollView’s contentView.

class GameViewController: UIViewController, UIScrollViewDelegate { @IBOutlet weak var contentView: UIView ! @IBOutlet weak var scrollView: UIScrollView ! ... }

Next well stub out our delegate methods and implement the viewForZoomingInScrollView(scrollView: UIScrollView) -> UIView? method.

class GameViewController: UIViewController, UIScrollViewDelegate { ... func scrollViewDidScroll (scrollView: UIScrollView) { } func scrollViewDidZoom (scrollView: UIScrollView) { } func viewForZoomingInScrollView (scrollView: UIScrollView) -> UIView? { return contentView } ... }

Sending data to the shader

A fragment shader can receive data from your Swift code via uniform variables. Uniform variables can be declared via the SpriteKit Editor. Let’s define some uniforms.

Open the GameScene.sks file and select the mandelbrot sprite. Scroll to the bottom of the inspector on the right and under “Custom Shader Uniforms” add 2 new items zoom of type float and value 1 and offset of type vec2 . We’ll populate this uniforms with the scrollViews contentOffset and zoom properties.

WARNING: Xcode 6.3 has a bug with uniform variables. Uniforms won’t be initialized with the values you provide in the editor. You’ll have to initialize them from code.

We can access the shader on our node via the shader property. To get a uniform from the shader we use the uniformeNamed() method. For example to retrieve our zoom uniform we use:

let zoomUniform = node . shader !. uniformNamed( " zoom " ) !

Once we have a uniform we can change its value via one of of the properties

var textureValue: SKTexture ! var floatValue: Float var floatVector2Value: GLKVector2 var floatVector3Value: GLKVector3 var floatVector4Value: GLKVector4 var floatMatrix2Value: GLKMatrix2 var floatMatrix3Value: GLKMatrix3 var floatMatrix4Value: GLKMatrix4

We’re only interestead in using floatValue and floatVector2Value for this tutorial.

Ex: to set the zoom to 2 we use

zoomUniform . floatValue = 2

Coordinate systems and mapping intervals

We’ll have to map between different coordinate systems while maintaining ratio. We’ll use this to convert the coordinates of our scrollview to the complex plane.

Lets first look at the 1D case:

To map a value x from the interval [0,a] to the interval [0,1] we just divide by the length of the interval x' = x / a .

To map a value x from the interval [0,1] to the interval [a,b] we multiply the value by the length of the interval and add the starting value of the interval to it x' = x * (b - a) + a

For example consider the x coordinate of an iPhone 4 in landscape. The x coordinate will be between 0 and 480. To map a value x to [0,1] we use x' = x / 480 . To map x' from [0,1] to [-2,2] we use x'' = x' * 4 - 2

If we have a point with x coordinate 120 on our screen the corresponding point in the interval [0,1] will be 120 / 480 = 0.25 and in the interval [-2,2] it will be 0.25 * 4 - 2 = -1 as seen below.

Mapping between the scrollview and the complex plane

We’ll have to convert points on our scrollView to points in the complex plane. The first step is to convert points from the scrollView to points in the range [0,1] . To do this we need to divide the contentOffset by the contentSize . This will bring our contentOffset to the range [0,1] .

var offset = scrollView.contentOffset offset.x /= scrollView.contentSize.width offset.y /= scrollView.contentSize.height

Our fragment shader provides points in the range [0,1] for both x and y coordinates which we’ll have to map to points that are in the interior of the scrollView’s contentView.

The normalized size of our contentView is 1.0 / zoom so the normalized coordinates of points in the contentView will be in the interval [contentOffset / contentSize,contentOffset / contentSize + 1.0 / zoom] .

One more thing that we have to keep in mind is that the y-Axis points upwards in GLSL and the point (0,0) is in the bottom left corner. We’ll have to flip the y-Axis so that it matches our scrollView.

The following GLSL code converts the position to a point in the scrollView’s contentView:

// Fractal.fsh void main { vec2 position = v_tex_coord; position. y = 1.0 - position. y ; // flip y coordinate vec2 z = offset + position / zoom; ... }

Below you can see in blue the frame of our scrollView’s contentView in both non-normalized and normalized coordinates. contentSize = (960,640) contentOffset = (240,160) , and zoom = 2.0

ScrollView

Normalized ScrollView

Finally we have to map our point to the complex plane. To get a good look at the mandelbrot set we’ll want to map to the region [-1.5,0.5] x [-1,1] in the complex plane.

We’ll also want to correct for aspectRatio. Currently Both our x and y coordinate have the same scale. We’ll want to scale our x coordinate by the aspect ratio so that the image isn’t distorted.

What is Aspect Ratio Aspect ratio is the ratio of a screen’s width to its height. [collapse]

// Fractal.fsh void main { ... z *= 2.0 ; z -= vec2 ( 1.5 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; ... }

Below you can see our scrollView’s contentView mapped to the complex plane and corrected for aspect ratio.

To integrate all the above code we’ll create a new method updateShader that will pass the coordinates of our contentView to the shader. We’ll need to call the updateShader method in the scrollview’s delegate methods.

class GameViewController: UIViewController, UIScrollViewDelegate { ... func updateShader (scrollView: UIScrollView) { let zoomUniform = node . shader !. uniformNamed( " zoom " ) ! let offsetUniform = node . shader !. uniformNamed( " offset " ) ! var offset = scrollView . contentOffset offset . x /= scrollView . contentSize . width offset . y /= scrollView . contentSize . height zoomUniform . floatValue = Float (scrollView . zoomScale) offsetUniform . floatVector2Value = GLKVector2Make( Float (offset . x), Float (offset . y)) } func scrollViewDidScroll (scrollView: UIScrollView) { updateShader(scrollView) } func scrollViewDidZoom (scrollView: UIScrollView) { updateShader(scrollView) } ... }

Also don’t forget to call the updateShader method when the view appears so that you initialize the uniforms.

class ViewController { ... override func viewDidAppear (animated: Bool ) { super . viewDidAppear(animated) updateShader(scrollView) } ... }

The shader code will finally look like this:

void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] position. y = 1.0 - position. y ; vec2 z = offset + position / zoom; z *= 2.0 ; z -= vec2 ( 1.5 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; vec2 c = z; float it = 0.0 ; // Keep track of what iteration we reached for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; if ( dot (z,z) > 4.0 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } vec3 color = vec3 ( 0.0 , 0.0 , 0.0 ); // initialize color to black if (it < float (iterations)) { color. x = sin (it / 3.0 ); color. y = cos (it / 6.0 ); color. z = cos (it / 12.0 + 3.14 / 4.0 ); } gl_FragColor = vec4 (color, 1.0 ); }

Complete Source Code

Challenges

1 . Optimization

The black areas of the set are the slowest to render. Luckily we can quickly determine if a point is in one of 2 large black regions (the cardioid or the region 2) as seen in the image below. Here you can find formulas for determining if a point is within one of these 2 regions. Improve the fragment shader by adding code that only performs the mandelbrot iterations if a point is outside these regions. This will greatly improve the apps performance when these regions are visible.

The main cardioid can be seen below in red and the region 2 in green.

Hint Perform the mandelbrot iterations only if the point is outside one of these regions. [collapse]

Solution void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] position. y = 1.0 - position. y ; vec2 z = offset + position / zoom; z *= 2.0 ; z -= vec2 ( 1.5 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; vec2 c = z; bool skipPoint = false ; // cardioid checking if ((z. x + 1.0 ) * (z. x + 1.0 ) + z. y * z. y < 0.0625 ) { skipPoint = true ; } // period 2 checking float q = (z. x - 0.25 ) * (z. x - 0.25 ) + z. y * z. y ; if (q * (q + (z. x - 0.25 )) < 0.25 * z. y * z. y ) { skipPoint = true ; } float it = 0.0 ; // Keep track of what iteration we reached if (!skipPoint) { for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; if ( dot (z,z) > 4.0 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } } vec3 color = vec3 ( 0.0 , 0.0 , 0.0 ); // initialize color to black if (it < float (iterations) && !skipPoint) { color. x = sin (it / 3.0 ); color. y = cos (it / 6.0 ); color. z = cos (it / 12.0 + 3.14 / 4.0 ); } gl_FragColor = vec4 (color, 1.0 ); } Complete Source Code [collapse]

2 . Make a similar app that lets you explore the Julia set for some point c .

Ex: vec2 c = vec2(-0.76, 0.15);

Solution void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] position. y = 1.0 - position. y ; vec2 z = offset + position / zoom; z *= 2.0 ; z -= vec2 ( 1.0 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; vec2 c = vec2 (- 0.76 , 0.15 ); float it = 0.0 ; // Keep track of what iteration we reached for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; if ( dot (z,z) > 4.0 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } vec3 color = vec3 ( 0.0 , 0.0 , 0.0 ); // initialize color to black if (it < float (iterations)) { color. x = sin (it / 3.0 ); color. y = cos (it / 6.0 ); color. z = cos (it / 12.0 + 3.14 / 4.0 ); } gl_FragColor = vec4 (color, 1.0 ); } Complete Source Code [collapse]

3 . Add the point c as a uniform allow the user to adjust it’s value by panning around with 2 fingers





Hint Use a UIPanGestureRecognizer to detect the 2 finger pan. You’ll have to normalize the translation that the gesture recognizer gives you [collapse]

Solution class GameViewController: UIViewController, UIScrollViewDelegate { ... var c: GLKVector2 = GLKVector2Make( 0 , 0 ) override func viewDidLoad () { ... let panGr = UIPanGestureRecognizer(target: self , action: " didPan: " ) panGr . minimumNumberOfTouches = 2 view . addGestureRecognizer(panGr) } func didPan (panGR: UIPanGestureRecognizer) { var translation = panGR . translationInView(view) translation . x /= view . frame . size . width translation . y /= view . frame . size . height c = GLKVector2Make( Float (translation . x) + c . x, Float (translation . y) + c . y) let cUniform = node . shader !. uniformNamed( " c " ) ! cUniform . floatVector2Value = c panGR . setTranslation(CGPointZero, inView: view) } } void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] position. y = 1.0 - position. y ; vec2 z = offset + position / zoom; z *= 2.0 ; z -= vec2 ( 1.0 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; float it = 0.0 ; // Keep track of what iteration we reached for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; if ( dot (z,z) > 4.0 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } vec3 color = vec3 ( 0.0 , 0.0 , 0.0 ); // initialize color to black if (it < float (iterations)) { color. x = sin (it / 3.0 ); color. y = cos (it / 6.0 ); color. z = cos (it / 12.0 + 3.14 / 4.0 ); } gl_FragColor = vec4 (color, 1.0 ); } Complete Source Code [collapse]

4 . Use an image to color a julia style fractal. There are many ways to accomplish this, an interesting one is using the following approach:

At each iteration get the color from the image coresponding to z . If the color is not transparent break out of the loop. If the obtained color is non transparent after running all the iterations use it to color the coresponding pixel If the obtained color is transparent use a different formula to color the point. for example the normalized number of iterations.

Here’s a julia set colored with the image of a bunny.

Hint You’ll have to add another uniform of type Texture named image . To get a color from the texture at position p you can use vec4 color = texture2D(image,p); [collapse]

Solution class GameViewController: UIViewController, UIScrollViewDelegate { ... override func viewDidLoad () { ... let imageUniform = node . shader !. uniformNamed( " image " ) ! imageUniform . textureValue = SKTexture(imageNamed: " bunny " ) } ... } vec4 getColor ( vec2 p) { if (p. x > 0.99 || p. y > 0.99 || p. x < 0.01 || p. y < 0.01 ) { return vec4 ( 0.0 ); } return texture2D (image,p); } void main () { #define iterations 128 vec2 position = v_tex_coord; // gets the location of the current pixel in the intervals [0..1] [0..1] position. y = 1.0 - position. y ; vec2 z = offset + position / zoom; z *= 2.0 ; z -= vec2 ( 1.0 , 1.0 ); float aspectRatio = u_sprite_size. x / u_sprite_size. y ; z. x *= aspectRatio; vec2 c = vec2 (- 0.76 , 0.15 ); vec4 color = vec4 ( 0.0 ); // initialize color to black float it = 0.0 ; // Keep track of what iteration we reached for ( int i = 0 ;i < iterations; ++i) { // zn = zn-1 ^ 2 + c // (x + yi) ^ 2 = x ^ 2 - y ^ 2 + 2xyi z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += c; color = getColor (z); if ( dot (z,z) > 4.0 || color. w > 0.1 ) { // dot(z,z) == length(z) ^ 2 only faster to compute break ; } it += 1.0 ; } if (color. w < 0.1 ) { float s = it / 80.0 ; color = vec4 (s,s,s, 1.0 ); } gl_FragColor = color; } Complete Source Code [collapse]

5 . Experiment with formulas for Mandelbrot like fractals This is a open ended challenge. 2 examples are provided below

Burning Ship Fractal

Formula z n = abs(z n-1 2 + c)

GLSL

z = vec2 (z.x * z.x - z.y * z.y, 2.0 * z.x * z.y); z += c; z = abs (z);

Source Code

Sierpinski Julia

Formula z n = z n-1 2 + 0.5 * c / (z n-1 2)

GLSLS

vec2 powc ( vec2 z, float p) { vec2 polar = vec2 ( length (z), atan (z. y ,z. x )); polar. x = pow (polar. x ,p); polar. y *= p; return vec2 (polar. x * cos (polar. y ),polar. x * sin (polar. y )); } void main () { ... z = vec2 (z. x * z. x - z. y * z. y , 2.0 * z. x * z. y ); z += 0.5 * c * powc (z,- 2.0 ); ... }

Source Code