Digitally Interfaced Book: Paper, Graphite, Makey Makey, Scratch, and Imagination
As a professional artist, deepening the ways in which seemingly disparate objects and processes are interconnected through locating, and mapping their intersections has been one of the main elements of my studio practice. The Fab Lab tools and working processes create an environment that is well suited to investigating those types of intersections.
In an effort to integrate Fab Lab tools centered on craft, and studio-based processes into the classroom, I have been working to implement a Mobile MakerCart at a project-based K-8 charter school. In addition to introducing craft-based physical computing projects to the children, a guiding principle behind the MakerCart is to give teachers the opportunity to develop familiarity with the MakerCart’s tools and processes in order to be able to envision the ways in which they might be able to develop their own curriculum for use in the classroom.
The cart, housing a laser cutter, 3D printer, sign cutter, sewing machine, and various circuit building components and tools, is a flexible platform designed to circumvent the school’s space issues, move the tools freely within the school, and also to fulfill one of the school’s missions of moving out and interacting with the broader community.
The initial project run through the cart was a DIY Sound Studio where the children learned the basic physics of sound, and learned to craft their own sound scapes. The project started with making lo-tech record players out of needles, solo cups, and chopsticks (Figure 1), and moved to crafting their own soundscapes in Audacity and Garage Band, and playing them with Scratch/MakeyMakey based instruments through speakers they made themselves (Figure 2). These projects are designed to be iterative and carried through into other projects to deepen and enrich the children’s processes of making, and the objects themselves.
A major component of the projects done by the children have been ones of engaging in reflective processes to help guide them toward developing studio habits of mind. Incorporating these reflective processes as part of the work itself, one of the next projects to be introduced through the MakerCart is the writing, and crafting of a book that will incorporate and be interactive with Scratch programmed environments through circuits built into the book. This project is intended to incorporate multiple elements of literacy through having the children craft richly layered narratives that interconnect across multiple media.
The Scratch programming language, developed by Mitchell Resnick and his team at MIT, allows children easier access to learning the principles of programming. Released in 2006, it is utilized extensively in both traditional school environments, as well as in informal learning spaces such as the Computer Club where it was first tested. Programming in Scratch is a process of connecting Graphical User Interface (GUI) drag and drop blocks into a work window. There are eight different types of blocks: Motion, Control, Looks, Sensing, Sound, Operators, Pen, and Variables which work in concert through their various commands. As a GUI programming environment the focus is taken away from needing to be syntactically correct in the programming process. This utilization of a GUI is intended to encourage a more playful process of developing programming literacy, by allowing for the programmer to move quickly through iterations of their building of the program (Figure 3).
With Scratch, young people can program their own interactive stories, animations, games, music, and art -- then share their creations with one another online. In the process, young people learn important mathematical and computational ideas, while also learning to think creatively, reason systematically, and work collaboratively. Scratch is designed to make the activity of programming more tinkerable, more meaningful, and more social -- and thus appeal to broader, more diverse audiences than traditional programming languages. Scratch builds on youth interests in popular culture, social media, and expressive communication.
Makey Makey is an object interface board developed parallel to, and intended to work in concert with Scratch. Makey Makey consists of a PCB that has all of the inputs that it takes to operate a computer keyboard, and a usb port that allows for the Makey Makey board to be connected to a computer. This feature allows for any keyboard command operation to be conducted through the Makey Makey board directly, but more importantly the board allows for the connection of any conductive object to run keyboard operations by connecting the objects directly to the Makey Makey board. Makey Makey boards are commonly demonstrated by connecting bananas to the board, and then using them as “piano keys” to play a virtual piano on the computer.
This book project uses all of the same basic parameters of hooking up the board to external objects, but utilizes a circuit that is drawn across the extent of the book in graphite (Figure 4). While the graphite is fussy at times, and will rub away leaving behind an inadequate substrate for the circuit to function properly, it is simple to develop the circuits, and also provides opportunity for the bookmaker to debug their book. Significantly, however, the book is a tangible object that is itself constructed, and done so in concert with programming in Scratch thus may provide a means to not only deepen their story in the book through external events in Scratch, but to also develop multiple literacies across media.
The media in this way are no longer abstracted from one another, nor is a story developed solely within a singular environment, either as only a book, or on the screen. Additionally the bookmaker has an opportunity to create his or her own visual and auditory environments in design programs such as Illustrator, Inkscape, Audacity, or Garage Band, among many others. This project can be seen through the lens of the remix technology being designed by Mozilla, and incorporated into such projects as the National Writing Project’s, Six Word Story, that is being run through Mozilla’s Thimble platform. A fundamental difference of this bookmaking project, however, is the physical component of it that allows for an embodied experience of the programming environment.
This opportunity to connect one’s body to the process as it relates to the programming environment is an extension of Seymour Papert’s ideas of embodied cognition as they relate to the LOGO Turtle. The bookmaking experience is clearly not a one to one body experience of course as with the movement of the Turtle as it relates to the body moving in space, but the aspect of creating the physical object that triggers the programmed environment through manipulation with the hands is an embodied experience nonetheless.
At the top of the post is a video documenting an initial test of the book-making process. It follows the arc of building events in Scratch, using a pencil to draw out a graphite circuit, and hooking it all up to a MaKey MaKey board to run the program through the graphite circuit.
1. INTERACTIVE BOOK ASSEMBLY
The following sections step out the process for creating the book circuit and hooking it up to the Makey Makey board to drive the Scratch programmed events.
1.1 Fold The Paper
Using a single piece of paper of any preferred size (here I have used 8.5” by 11”) follow the instructions for folding and cutting the paper as provided in Figure 5.
1.2 Create Your Story
Using your preferred creative method for crafting a story in text or image, or both, fill in the pages of your book with something amazing, hysterical, historical, or otherwise as in Figure 6.
1.3 Build The Circuit
Using graphite, or any other conductive material (here I have used aluminum tape), build a circuit that will allow an action in Scratch to be triggered from each page you want to have interactive, as shown in Figure 7.
The integrity of the book’s circuit is important because only one ground wire will connect the book to the Makey Makey board, and then a jumper wire will lead from each circuit point on the bottom of each page to the respective keyboard key interface on the Makey Makey board as will be shown in the steps provided in the following section.
1.4 Hook Up Your Book to Makey Makey
Taking one point as your ground on the book circuit, run a jumper wire to the Earth Strip on the Makey Makey Board. Using jumper wires continue to hook up each other point in the circuit on the book to the section of the Makey Makey board that you have programmed Scratch to be triggered by. In this example the keys I am using are Space, Up Arrow, Down Arrow, and Right Arrow, as shown in Figure 8 on the following page.
1.5 Plug in Makey Makey and Run Scratch
Plug the Makey Makey board into the computer, and load your Scratch program (Figures 9 and 10).
1.6 Read Your Book, Make Scratch Play
Thumb through your book and use your finger to close the circuits. Your body is the switch, and the book circuit should now run through the Makey Makey board and trigger the events you’ve programmed in Scratch as shown in Figure 11.