In this short project, which I finished in December of 2017, I took colour photos and represented them using images of skittles. To start off, I found photographs of skittles on the internet and segmented out the skittles manually. I was going to use my own images, but there are some flavours of skittles, which I couldn't find at the time, and I wanted to get started right away. Since then, I have produced a library of my own skittle photos.
There were I found eight skittle colours in total: blue, green, orange, pink, purple, white, yellow, and red. The purple skittles are very dark and look like black in contrast to the other colours. In a short python programme, I extracted small chips of an input photo and compared it to the images of the skittles. The programme then rotated the skittle image through 90, 180, and 270 degrees to see which was were best fit for the photo chip. These skittle images were then saved.
I quickly swapped to using my own skittle images, where possible. On a sheet of white paper and with a flash I took some photos of the contents of a small packet of skittles. For some reason, after each shot, the number of remaining skittles decreased. In another short python script, I used the open computer vision library to segment out the skittles and save them. I used a few steps to do this. I collapsed the images to greyscale and then blurred it and thresholded the image to produce a black and white image. This created white and black regions which corresponded to the location of the skittles, but also found regions in the background made up of noise. I then used two morphological operations, an erosion followed by a dilation. Erosion makes make black regions smaller by a set amount, and dilation makes those regions larger again. This is very useful for removing small regions in image processing tasks, like noise. Noise regions are often small enough to be removed completely when eroded, and the larger skittles are only reduced in size. I then found the location of the edges of the black regions and used this to cut out the skittles.
So, how delicious would portraits made out of skittles be? Well, I don't know, as I never made any out of actual skittles. To get a decent, recognisable image you need about 10 kS (that's kiloSkittles), or around 100 skittles on the short edge. Since skittles are about 1cm wide, it would be quite a large structure. On top of that, skittles also weigh about a gram, which would mean you'd need ~10kg of skittles, and at a penny a sweet you'd spend £100. That assumes that you can bulk buy the correct colours, which isn't the case, most of the images I produced are mostly purple or yellow, with very little green.
Bulk buying individual colours would make it feasible, but still not trivial. And then there's the placing of >10 kS, although I can see that being quite therapeutic. I assumed that setting them in resin would be sufficient to preserve them, their high sugar content and a lack of oxygen should probably stop them from changing much over months, but I don't know what might happen over a period of years.
If I ever revisit the idea I'll add a hexagonal stacking mode, as well as the option to include dithering for greater colour accuracy. I have already added functionality for rectangular images of skittles from the side, to increase the resolution of the image in one direction. This worked, but the results weren't as pleasing. I imagine, with the skittles on their sides, it would be that much harder to assemble.
Dithering is an interesting concept. This medium renders images in an unusual way. If we forget about photons, the intensity of light on a given area is a continuous property, it could be twice as much as another area, or half as much, or any fraction. Digital cameras quantise this level when the analogue signal (a voltage) is converted to a digital level in a range. On dSLRs this is range is typically 12-14 bits or between 4096 and 16,384 levels. This is often then down-converted to 8 bits (256 levels) if the image is saved as a jpg, but the full range is sometimes kept with other file types.
256 intensity levels for each colour channel is sufficient for viewing or printing images. The ability of the eye to differentiate between intensity levels is dependent on a large number of factors, including the absolute intensity of the source, as well as the colour. Colour photos allow the representation of a large number of colours and shades, typically 16.8 million. These colours can be represented in Hue, Saturation, Value or HSV space. HSV breaks down a pixel into its 'value', the brightness of the pixel, 'hue' the pure colour of the pixel, and 'saturation' which is how vivid the colour is, or how much of the pure colour is mixed with white. Now, our skittles well sample the different hues which are in photos, but they do not well sample any of the other dimensions of HSV space. Purple skittles are the only dark skittles, and the rest of them are all highly saturated bright colours, save for white, which is still bright. Because of this, many images cannot be well represented by skittles (I know, who'd have thought that?).
This is where dithering comes in. Dithering, in this case, trades of spatial resolution for better colour representation. A small image patch is considered, and the average colour in that patch is matched to several skittles. A light pink area, without dithering, would be represented as (say) four white skittles. But with dithering, one or two of the skittles could be swapped to red or pink. When viewed from a distance this would look much like a lighter pink. This would allow for areas of tone which are all similar to skittle colours to show some detail, and also allow for more accurate representation of the colour of image regions which are not well represented by skittle colours.
Dithering does have some disadvantages. You lose spatial resolution, as boundaries between colours are blurred to intermediate values. It also doesn't make 'sense' when viewed close up. A flat light pink region of an image may have a bright red skittle in the middle of it, and it might not be obvious from the original image why it is there.
I quite like them, anyway.