Blog | Allie Jones

Migrating from MAMP to Homebrew

Fri, 15 Feb 2013 16:00:00 GMT

I've been using MAMP (the free version, not MAMP Pro) on my local machine for development and testing for a long time, mostly out of laziness. FuelPHP's oil utility does not work well with MAMP's MySQL, however, so today I decided it was finally time to switch to using homebrew-installed MySQL and PHP and OSX 10.8's included Apache.

It was generally painless (I used these instructions and this blog post about enabling Apache on Mountain Lion) but I didn't want to have to recreate all of my local databases. Some Googling only turned up the location of MAMP Pro's database files. I found the files for regular MAMP at /Applications/MAMP/db/mysql/ and moved them to /usr/local/var/mysql/ and everything was good to go.

While researching this, I also came across this blog post explaining a setup that automatically makes each folder in your Sites directory available at <foldername>.dev and set that up as well. It's awesome. No more editing my hosts file and Apache config!

A new blog

Fri, 15 Feb 2013 16:00:00 GMT

One of my recent personal goals has been to get more comfortable with writing about technical and programming topics. Starting a new blog seemed like a good way to practice this.

So naturally I got totally sidetracked by choosing the right platform, trying to port over my portfolio to a new format and agonizing over what kind of design it should have ... which is why I haven't actually written anything, despite intending to for months now.

However, for the next three months I am fortunate enough to be attending Hacker School, where I have the freedom to learn about whatever cool and interesting programming topics I want---things I should definitely write about! So no more procrastination; Octopress' default template it is.

I've been in New York and at Hacker School for a week now and so far things have been awesome. More thoughts on that later!

JavaScript function statements vs. function expressions

Tue, 19 Feb 2013 16:00:00 GMT

I was talking about JavaScript scope today (and how, when first learning the language, I assumed JS had block rather than functional scope and was often confused). That reminded me of a seemingly-strange Javascript behavior I had discovered previously.

When declaring functions, this works:

function two() {
  one();
}
two();
function one() {
  console.log("one!");
}

But this does not (throwing the error one is not a function):

var two = function () {
  one();
};
two();
var one = function () {
  console.log("one!");
};

(I had trouble finding a concise way to describe these two methods of creating a function, but JavaScript: The Good Parts enlightened me: function statements and function expressions.)

This behavior happens because when JavaScript initially parses a file, it evaluates all of the function statements first, before executing anything. So a named function (a function statement) is always available before any code is executed, regardless of where the function appears in a file. An anonymous function assigned to a variable is only available after the variable is defined and assigned the function.

Daniel Mendel, fellow Hacker Schooler, talked through this with me and also wrote a blog post about it.

HS Project: Makefolio

Fri, 01 Mar 2013 16:00:00 GMT

For my first project at Hacker School, I decided to build a static site generator for portfolios. (Wondering what a static site generator is? Here's a pretty good explanation.)

I have attempted to migrate my portfolio to a few different static site generators without success. (I think the one I got farthest with was Punch, but I still ran into trouble.) My format of mostly images with a small amount of text was just not the sort of the website most generators' authors had in mind, so trying to shoehorn my existing website into their format was difficult.

Writing a site generator that handled collections of images well seemed like both a way to solve my problem and a perfectly-sized challenge. I wasn't totally sure how to accomplish it, but I had a general idea and a specific end goal in mind: being able to generate my current site using the tool I would write.

My goals

Get more comfortable with Ruby

Prior to this project, I'd written enough Ruby to know that I liked it, but hadn't built anything substantial. This was a great way to get more familar with the language (especially without the added complexity of Rails). Having other Hacker Schoolers review my code and suggest more Ruby-like ways to do things was incredibly helpful too.

Gain more experience structuring applications

Most of my previous programming experience involved working with pre-existing frameworks, and I want more experience making the structural decisions neccessary when building software from (mostly) scratch.

Try test-driven development

I think I'm a convert! Though I felt like I was spending far too much time writing tests and trying to understand Rspec at the beginning of the project, by the mid-point I was so glad I had taken the time to write them. It felt great to be able to dramatically refactor things and know I hadn't broken everything. (A big benefit to my other goal of getting more experience organizing my code!)

Better understand how templating works

I have more to do on this one. I used basic ERB templates, but I can't say I really understand what is happening when I pass a Ruby Binding object to a template. Something to investigate further.

Other things I learned

This was the first time I've tried to write a thorough README for something I've made, and I was surprised at how well it clarified what I wanted Makefolio to do. Multiple features and improvements occurred to me as I was writing. (I now have a better appreciation of Amazon's press release first method of product development!)

Opening the box: things I learned while learning about sockets

Thu, 21 Mar 2013 16:00:00 GMT

One of the pieces of advice we were given at the beginning of Hacker School was "open the box"---that is, don't blindly use things in your programming projects without any idea of how they work. Don't gloss over things you don't understand, take the time to learn more about them. Feeling encouraged to really deepen your level of understanding (and having the time to do so!) is one of the best things about Hacker School's focus on learning and becoming a better programmer.

One of my Hacker School goals is to fill in the gaps and fuzzy patches in my understanding of the web stack and how the Web works. Towards that end, this week I read through Jesse Storimer's Working with TCP Sockets.

I kept track of all of the unfamiliar topics I found myself investigating further while reading through the book and writing my own code based on his examples. I'd probably never finish this post if I wrote out explanations of each of these things, but even just the list is interesting.

Things I learned while learning about sockets

the TCP urgent flag (that no one ever uses)
some of the differences between the MRI, JRuby and Rubinius implementations of Ruby
Ruby functions that were new to me: pack, unpack, exit
C's fixed-width integer data type
binary representation of text
the concepts big-endian and little-endian
4 bits (half a byte) is a 'nibble' (Cute.)
the difference between threads and processes
the FTP specification
POSIX signals (and capturing them in Ruby scripts)

In the same vein as this, Sasha Laundy has a great post about how she learned all kinds of things while trying to fix a bug in Octopress.

Missing dependencies with Heroku, Python and pip

Thu, 28 Mar 2013 16:00:00 GMT

I've been experimenting with some machine learning techniques after Hilary Mason spoke about it to Hacker School last week. I'll save discussing exactly what I've been working on for another post, but I've been programming in Python to take advantage of the libraries available to assist with the math involved.

I want to build some web applications based on my scripts, so I've been working on a bare-bones Flask app to provide their results in json format. Getting a server running on EC2 proved beyond my sysadmin abilities and patience at the moment, so I decided to deploy to Heroku instead.

My script uses the hcluster library, which depends on numpy, and this is where I ran into trouble. When I pushed my application to Heroku, it would attempt to install the packages in pip's requirements.txt. hcluster would fail to install because it couldn't find numpy, even if pip had already attempted to install numpy (or so I thought). This happened regardless of the order of the packages in the requirements file.

The solution turned out to be removing hcluster from requirements.txt temporarily, pushing to Heroku so numpy would be installed successfully, then restoring hcluster to requirements.txt and pushing again. Not the most elegant fix, but it worked. This StackOverflow question about a different library pointed me towards this solution.

It turns out that pip first runs each module's setup.py, then installs. (So I thought numpy was being installed first, but it actually wasn't.) Apparently checking for module dependencies in the setup script is an incorrect use of setup.py, but it seems like hcluster isn't the only library to do so. There's also a Github issue for pip that provides more context.

SPACEJUMP

Sun, 31 Mar 2013 16:00:00 GMT

A Hacker School facilitator recommended Michal Zalewski's The Twisted Web to me, and I've really been enjoying it. I'd say it's the most entertaining highly technical book I've ever read, and it is densely packed with great information about how the Web works.

However, I was reading it this afternoon and was distracted from its more substantial content by an offhand comment about the HTTP SPACEJUMP method. I was offline right then (thanks, laundromat with crappy cell reception) and couldn't look up its actual use, but it seemed like a charming Easter egg in the HTTP protocol.

Sadly, SPACEJUMP has nothing to do with astronauts as I was imagining. It was intended for something like an image map.

I'm a little disappointed. HTTP response code 418 (I'm a teapot) will have to remain my favorite bit of web protocol silliness.

Use your Github code as a website background

Sun, 31 Mar 2013 16:00:00 GMT

I've been thinking about the best way to add my Hacker School projects to my portfolio. Prior to the past six weeks, I haven't written much code I could share publicly, and most of my projects included enough design that it wasn't difficult to create a representative screenshot. That's not the case with everything I'm working on now.

Displaying the source code itself seemed like a good possible solution---maybe not quite as interesting as images, but not bad. But I didn't want to have to copy and paste code or clutter up my HTML with what would be essentially decorative text.

The final product

<pre class="codepen" data-height="300" data-type="result" data-href="vExdA" data-user="alliejones" data-safe="true"><code></code><a href="https://codepen.io/alliejones/pen/vExdA">Check out this Pen!</a></pre> <script async src="https://codepen.io/assets/embed/ei.js"></script>

Loading the source code

My first thought was to use a <script> tag, similar to the way JavaScript templates can be embedded in a page. Something like this.

<script type="text" src="/path/to/code.rb"></script>

That doesn't work, however. JavaScript can't access the script tag's contents unless the code is embedded in the page, rather than linked, so I'd still have to copy and paste. (There may be other problems with this approach, too, but I didn't pursue it any further.)

I then realized that Github has an API, and most of my code is already hosted there, so I wouldn't even have to do any extra work to make the source accessible. Perfect.

Github's API supports JSONP, so you can access it via JavaScript from any domain. (I just learned recently that jQuery includes support for JSONP AJAX requests. It handles the callback function for you so that a JSONP request can be treated like a normal AJAX request.)

$.getJSON(
  "https://api.github.com/repos/alliejones/makefolio/contents/lib/makefolio/project.rb?callback=?",
  function (response) {
    var code = Base64.decode(response.data.content);
    $("div").attr("data-code", code); // see below
  }
);

The one complication is that the file contents are base-64 encoded in Github's response, so I had to find a JavaScript base-64 decoding library that supported utf-8. (Only Firefox has built in functions to encode/decode base-64 data, and even it requires a workaround for utf-8.)

Getting it on the page with CSS

Using a pseudo-element seemed like the best approach in this case, since I'm creating a largely decorative effect and trying to avoid having to change the structure of my HTML.

By combining the attr() CSS expression, CSS generated content and a custom data attribute, you can set the contents of a pseudo element to pretty much any text.

When attr() is applied to a pseudo-element, its context is the pseudo-element's parent element, so the data attribute should be set on the original, content-containing element. The custom data attribute doesn't even need to exist in the HTML---it can be added via jQuery, as I'm doing in my AJAX request callback.

$("div").attr("data-code", code);

(An aside: jQuery includes a data function that can be used to access custom data attributes that have been set on HTML elements, but using that function to set custom data doesn't update the DOM, so it won't work in this case.)

With a few additional CSS properties to set the appropriate monospaced font and place our background text behind the actual content, this is ready for whatever additional styling you might like to add.

div {
  position: relative;
}

div:after {
  content: attr(data-code);
  font-family: monospace;
  top: 0;
  left: 0;
  max-height: 100%;
  max-width: 100%;
  overflow: hidden;
  position: absolute;
  white-space: pre;
  z-index: -1;
}

It isn't possible to have syntax highlighted code with this particular CSS approach, because the content property only supports plain text. I'm okay with single-color code in this case, and didn't want to deal with integrating a code highlighting library as well, but you could probably wrangle something by adding the extra HTML elements containing your highlighted code to the page via JavaScript.

Additional considerations

Don't use Github as a CDN

The first version of my sample on Codepen was broken in Chrome Canary for a reason unrelated to what I was ultimately trying to achieve. Out of laziness, I was linking to a base-64 decoding library hosted on raw.github.com, and Canary refused to load it.

Github serves raw source code files with the MIME type of text/plain and the HTTP header X-Content-Type-Options: nosniff. That header was new to me, but it is basically a request from the server that browsers accept the MIME type specified, rather than trying to guess what type of file they may be receiving. The end result is that browsers that respect the nosniff header won't execute JavaScript code sent as text/plain. (Github does this because they'd prefer you didn't hotlink to raw source code in a repository, which seems pretty reasonable.)

This is a recent change in Canary that will eventually make its way to Chrome. IE already supports nosniff and Firefox is planning to do so. There's a Chromium issue that explains further.

Browser support and accessibility

Browser support for this isn't bad (Webkit, Firefox, IE 10), though since the source code is loaded asynchronously (and therefore doesn't block the page from rendering) and this is really just a cool detail rather than critical content, it degrades nicely in older browsers.

Github's API is only available over HTTPS, so some browsers may refuse to make the AJAX request if the initiating page was loaded over HTTP. (This may be why it doesn't work in earlier versions of IE, but I'm not sure.)

CSS generated content is printed by default, so the code should probably be hidden in a print stylesheet. Apparently screenreaders' behavior with pseudo-elements varies widely, so I'm not sure what the best solution to that would be.

Getting started with Grunt

Thu, 04 Apr 2013 16:00:00 GMT

I love Grunt. I wouldn't want to work on a JavaScript project much bigger than a file or two without it, now that I've realized how much time and hassle it saves me.

But getting started using it involves a lot of friction. Even though I'm familiar with the tool I still drag my feet setting it up on a new project.

Yesterday, so that I would no longer have this problem, I created a template Gruntfile for myself. It contains the tasks I use on nearly every project:

JavaScript linting (with JSHint)
file concatenation and compression (using Uglify)
file watching (so that I don't have to invoke Grunt manually every time I change a file)
Growl notifications for warnings and errors (so you don't have to waste screen space keeping your terminal visible)

With the exception of Growl, these are all official Grunt packages.

This would be a great starter configuration for someone new to Grunt, and once you have this setup working it is easy to find and add new tasks. Using it is pretty straightforward, but I'll explain the important details below. (I've also saved this file as a gist if you want to save your own changes.)

module.exports = function (grunt) {
  var defaultTasks = ["jshint", "concat", "uglify"]; // used for watch as well

  var files = [
    /* your source files */
  ];

  grunt.initConfig({
    jshint: { all: files.concat(["Gruntfile.js"]) },

    concat: {
      all: {
        src: files,
        dest: "project.all.js",
      },
    },

    uglify: {
      all: { files: { "project.min.js": files } },
    },

    watch: {
      all: {
        files: files,
        tasks: defaultTasks,
        options: { debounceDelay: 250 },
      },
    },
  });

  grunt.loadNpmTasks("grunt-contrib-concat");
  grunt.loadNpmTasks("grunt-contrib-uglify");
  grunt.loadNpmTasks("grunt-contrib-watch");
  grunt.loadNpmTasks("grunt-contrib-jshint");

  grunt.registerTask("default", defaultTasks);

  var growl = require("growl");

  ["warn", "fatal"].forEach(function (level) {
    grunt.util.hooker.hook(grunt.fail, level, function (opt) {
      growl(opt.name, {
        title: opt.message,
        image: "Console",
      });
    });
  });
};

A note: These instructions assume you are using OSX.

Requirements

Grunt requires Node.js and npm to be installed (npm, Node's package manager, is included in the Node installer).

Grunt comes in two parts, one that only needs to be installed once and a second that is installed for each project.

The first time you set up Grunt, you should run npm -g install grunt-cli. This makes the grunt command available on your system. (More information here.)

Then, from the root directory of the project you'd like to use Grunt with, run the following to install the packages you need via npm:

npm install grunt
npm install grunt-contrib-concat
npm install grunt-contrib-uglify
npm install grunt-contrib-watch
npm install grunt-contrib-jshint
npm install growl

The Growl notifications also require that Growl be installed, as well as the Growl command line tool growlnotify (downloadable here).

Configuration

You should save the Gruntfile to your project's root directory with the filename Gruntfile.js.

Gruntfiles are just JavaScript, so you can use variables and functions as you normally would. I've set up this Gruntfile so that you only have to specify the locations of your project's JavaScript files once, at the top of the file.

var files = ["main.js", "src/**/*.js"];

This array of file paths will be passed to each Grunt task automatically. Grunt supports glob paths as well (so src/**/*.js will include every file with the extension .js in any subfolder of src).

If you need specify files in a more particular fashion, the supported syntax is extremely flexible and powerful, but I rarely find myself needing that much complexity.

To actually use the tasks you've configured, run grunt watch from the root directory of your project. Your JavaScript files will be automatically checked for errors, concatenated and minified each time you modify a file, and your JavaScript build process will be much less tedious!

Full-bleed images with TCPDF

Fri, 07 Jun 2013 16:00:00 GMT

The current PHP project I'm working on needs to generate PDFs, and I'm using the TCPDF class to do so. TCPDF, while powerful, has documentation that is ... not very user-friendly, to put it mildly.

I ran into problems trying to display an SVG file as a full-bleed image (that is, I didn't want white space between the edges of the PDF document and the image). I would set the document size and the SVG display size to the same values, and set the document margins to zero, and yet, infuriatingly, my SVG file would display just slightly smaller than the full page size.

After some searching and experimentation, I found another margin property that needed to be changed. To remove all of the margins on a TCPDF-generated PDF file, you need to set the bottom margin of each page via the setPageOrientation method as well as setMargins.

The bottom margin was creating whitespace on the file's right edge as well, because the SVG file was maintaining its original aspect ratio.

The full code to remove all of the margins on the page was this (the header and footer also have to be disabled):

<?php
$pdf = new TCPDF(/* ... */);
$pdf->setPrintHeader(false);
$pdf->setPrintFooter(false);
$pdf->setMargins(0, 0, 0, true);
$pdf->setPageOrientation('', false, 0);
// the third argument above is the bottom margin size

The old year and the new one

Sun, 19 Jan 2014 16:00:00 GMT

2013 was probably the most eventful year I've had in my post-school life: in the space of ten days this past August, I got married, moved to a new (big!) city and started a new job as a "real" programmer.

Though it seems implausible even to me, I didn't have any of these things planned at the beginning of 2013. I applied to Hacker School on a whim around New Year's 2012, and deciding to attend set all of these things in motion. (Well, that, and the Rhode Island legislature finally getting their act together on gay marriage.)

I never did write a substantial Hacker School reflection post, but I am without question a better and more confident programmer, I have a bunch of new friends and my new job (at Photoshelter) makes me much happier every day. I'd definitely call that as a success.

I'm not one to talk about my personal life on the internet much (at least now that the livejournal chronicling my teenage years is gone, hah), but beyond all of the awesome tech-related things that I accomplished this year, getting to marry my favorite person was still the best.

I expect 2014 will be a little quieter, and I'm more than content with that, but I have put some thought into things I'd like to do this year. I'm being not-particularly-ambitious in the hopes I'll be more likely to actually accomplish everything.

Finish reading and working through The Elements of Computing Systems (aka nand2tetris)
Work through at least a few chapters of SICP
Build some toy projects with AngularJS and React, and possibly give Clojure/ClojureScript a try
Blog more (I'm doing a friendly blog-post-a-week competition, starting this week, so I hope it will help!)
More generally, do more substantial things and fewer bite-size things (e.g.: less Twitter, more books; less tiny iOS puzzle games, more getting through the huge backlog of things I would like to play ...)

I was thinking along these lines before I stumbled upon this blog post, but Michael Lopp's essay The Builder's High is a more eloquent explanation of my fuzzier thoughts.

An art school lesson

Sun, 02 Feb 2014 16:00:00 GMT

The first studio class I attended my first week of college was three-dimensional design. I was extremely nervous, and things started strangely. The professor was absent, and had left a TA who instructed each student to each choose an art postcard from a collection pinned to the wall, with no further guidelines.

Once everyone in the class had chosen a postcard, we were told our assignment was to recreate these paintings in three dimensions. Newspaper and tape were distributed. We had a few hours to put together a rough proposal of our ideas for how to build our projects.

I can't remember the artist or title of painting I chose, though they were all abstract or stylized in some way that made translating the image to three dimensions challenging.

What I do remember is what the professor said to me once I had hesitantly presented my idea to him: "That looks great. But you should make it twice that size." A simple suggestion, but an intimidating one for me--I hardly knew how I was going to build this project at all, let alone at a large scale. But he was the professor, so I took his advice.

There wasn't much for me to do after that except start working, so I did. What I realized is that you don't have to know how the entire thing is going to come together when you begin something: just start with a small part, learn as you go, and build from there. It wasn't completely smooth sailing, but my project turned out great, and I felt awesome about making something much more ambitious than I thought I could.

I still think about that experience often when I am starting something new. "Great, but make it bigger!" isn't useful advice for everyone, but for me, as a person who tends to be more pragmatic and conservative than I need to be, it makes a big difference.

I don't make ambitious art projects as often anymore, but I find "make it bigger!" applies well to programming, too.

When I started my collaborative drawing app, I barely knew how any of the parts would work. Handling real-time events seemed intimidating, and so did working with TCP sockets. But I figured I could learn enough to build a tiny terminal-based chat, and I did. Once that existed, rewriting what I had using EventMachine seemed straightforward, and then it didn't seem so difficult to switch to using websockets instead.

Setting a goal that seemed slightly scary and working towards it a small piece at a time got me much farther than I would have thought I could at the beginning. An excellent thing to remember any time a new programming task or skill seems daunting.

Javascript: String.replace()

Sun, 09 Feb 2014 16:00:00 GMT

I've always used String.replace() in Javascript pretty much like this:

"Some <foo>".replace("<foo>", "bar");
// 'Some bar'

Or with a regular expression if I was feeling fancy or the pattern to match was more complex:

"Some <foo>".replace(/<\w+>/, "bar");
// 'Some bar'

"Some <baz> and a bit of <foo>".replace(/<\w+>/g, "bar");
// 'Some bar and a bit of bar'

What I didn't realize until this week is that you can also pass a callback function to replace(), which makes it possible to perform more complicated (and also possibly easier to follow) manipulation of the matched portions of the string before they're replaced.

You could even use this to create a super simple (and, sure, probably inadvisable) templating system:

var variables = { foo: "one", bar: "two", baz: "sea" };
var template = "{{foo}} if by land, {{bar}} if by {{baz}}";

template.replace(/\{\{(\w+)\}\}/g, function (match, group, offset, string) {
  return variables[group];
});
// 'one if by land, two if by sea'

The function is called once for each match within the string, and each parenthesized group in the match gets passed as a parameter to the callback function (which does mean the number of function parameters is variable, based on how many groups you have in your regular expression). Here I've used grouping to pull out the variable name inside the {{ }} delimiters and use it as an object property.

Not a Javascript feature I will use every day, but definitely one that will come in handy occasionally (and most likely be much cleaner than whatever code I might have written without it).

The best games I've played recently

Sun, 16 Feb 2014 16:00:00 GMT

Though I will happily play a typical AAA-style video game when an appealing one comes along, most of my gaming time recently has been spent playing casual puzzle games and more experimental things, so I've divided my list into those categories.

Puzzles

Threes iOS
Beautiful design, simple rules and surprisingly deep strategy.

Blendoku iOS, Android
Crossword puzzles that use colors instead of words. Like color theory excercises turned into a surprisingly fun game.

Enjoy Sudoku iOS, Android, OSX
This isn't a particularly attractive version of Sudoku, but it's become my favorite. It's the only Sudoku app I've found that helps you learn more complex solving strategies.

Experimental Narratives

Blackbar iOS, Android
A game told through censored letters. Puzzling out what words have been blacked out to advance through the story is surprisingly compelling.

Device 6 iOS
A beautifully designed short story turned into a puzzle game. Sometimes frustrating, but interesting and clever. The developer Simogo's other game Year Walk is very different in presentation (more like an adventure game) but also worthwhile.

Brothers: A Tale of Two Sons PC, PS3, XBox
An interesting take on tightly tying a game's control scheme and story: you play both title characters at once, controlling one brother with each analog stick. I loved the dark-but-beautiful fairy tale setting, and though the plot was a little heavy-handed it was still affecting.

999: 9 Hours, 9 Persons, 9 Doors 3DS
A visual novel and escape-the-room game hybrid. The plot is ridiculous, but I found the game enjoyably suspenseful and creepy regardless.

Thoughts on debugging JavaScript, part one

Sun, 23 Feb 2014 16:00:00 GMT

Spending the past few months becoming familiar with a large and complex JavaScript codebase has been a learning experience. I have had a few tearing-my-hair-out moments of frustration, but I've also become a much more systematic debugger. In the hopes it might be helpful, and also to clarify these things for myself, here are some notes on the topic.

It turns out I have a lot of feelings about browser devtools, so that's going to be Part One. Part Two will be less specific to debugging JavaScript in the browser, and hopefully more generally applicable.

Befriend your developer tools

Seriously. There are so many useful things hiding in there you probably haven't discovered yet. I'm not such a devotee that I use the devtools as my editor (maybe someday), but I do pretty much always have them open while I'm working.

I prefer the Chrome tools, mostly arbitrarily, but Safari and Firefox have very similar features. CodeSchool has a free Chrome devtools course that's helpful for getting started.

In my own debugging, I use the Elements, Network and Console panels most heavily (learning the profiling tools better is on my to-do list).

I think the Elements panel is pretty straightforward and self-explantory, but the Network and Console panels have a lot of hidden functionality you might be unfamiliar with. (And I'm sure they are plenty more things I haven't discovered yet either.)

Network

The Network panel is useful for answering questions like, "Did this ajax request even happen?" or "What data did I just post to the server?".

Clicking on a filename in the Name column will open a detail pane with lots of information about your request. It is usually hidden by the expanded-by-default Request Headers section, but the form data passed with the request (something I often want to check) is recorded here. Viewing the server's response (helpfully under that Response tab) can be useful when debugging too.

Also, I'm filtering the requests by type in this screenshot, limiting it to only ajax/XHR requests. This helps a lot with finding the actual requests you're interested in, since I don't particularly care about images if I'm trying to figure out why an ajax request is broken.

It took me far too long to realize that if you hover over the filename in the Initiator column, you can view a stack trace that shows where an ajax request came from. So here's a screenshot of that, too, so you don't have to discover it accidentally like I did.

Console

I use the console a lot for trying out and troubleshooting small pieces of code, but it can also be useful if you're trying to understand an unfamiliar codebase.

If there is a class (or jQuery plugin, or whatever) whose behavior seems mysterious, trying to use it and poking around in its properties from the console can often help clarify things. (If the class in question is inside a module or otherwise out of scope, I'll edit the code temporarily to set it as a property of window, so it will be accessible from the console.)

Extras

The debugger keyword is incredibly useful, and often better than using console.log (though I use that often, too) or trying to set breakpoints in large files. It will pause code execution wherever you put it, allowing you to see the current state of the application and step through what is happening.

When paused in the debugger, the "Continue to here" option can be useful for jumping into callback functions (which can be tricky or tedious to do otherwise). That option is available by right-clicking on the line number you'd like to stop at.

console.trace can also come in handy. It is similar to console.log, but prints a stack trace to the console instead of its passed arguments.

More thoughts (and less devtools nerdery) next week!

Thoughts on debugging JavaScript, part two

Sun, 02 Mar 2014 16:00:00 GMT

import { Link } from "gatsby";

Following up my dev tools lovefest, I have some more advice on debugging JavaScript, including some approaches I find helpful and editor features worth learning. First, some strategy:

Develop a theory

Starting the process of tracking down a bug can be overwhelming if you're working on code you're unfamiliar with, either because you've never seen it before or because you wrote it months ago and have completely forgotten about its details. It is easy (for me, anyway) to feel paralyzed by not even knowing how things were intended to work or how all of the pieces fit together, let alone what component is now causing problems.

One approach I find helpful in this situation is to invent a theory about what's causing the bug, and attempt to confirm or disprove that specific explanation. I say "invent" because, while a plausible theory helps, pretty much any reasonable one will suggest some concrete details of the code you need to investigate.

For example, say I've been given a bug where a particular widget in the application isn't rendering. My first theory might be, "Maybe the ajax request to the API isn't returning data in the correct format", or "I wonder if the render function isn't getting called for some reason", or "My coworker made some changes to the template function recently, something weird could be happening there".

The point isn't to guess correctly on the first try, but instead to start digging through the low-level details of the code. It is much easier to do that with a working theory that suggests specific questions like, "Where in this code does the widget's render function get called?" or "What data format is this widget expecting?". You might not have been (probably weren't, even) right, but in finding that out you've learned a lot about how everything works, and you're better positioned to develop your next theory, which will likely be more accurate.

A bonus benefit of this approach is that as you continue to use it and your familiarity with the code you're working on increases, your intuition about what is causing a bug improves, and your first guess will be right more often.

Break things

The code you're working on is (I sincerely hope) under version control, which is awesome, because it means you're free to break everything in your quest to fix things. Restoring order afterwards is as easy as running git reset.

When I first started working on a production application with a team of other developers, I was afraid to touch core parts of the code, because what if I ruined everything?! But don't be timid—just because committing that change would break all kinds of things doesn't mean you can't do it in your development environment.

When you're learning a complicated system, it can really help a lot to sprinkle around console.logs with abandon, or comment out a critical function call just to see what fails. Sometimes I will comment out everything and try to "rebuild" it by re-enabling components piece-by-piece, to better understand how the parts work together and depend on each other.

As a caveat, this does require some knowledge of which parts of your dev environment might be shared or not revertable (there may be things you really shouldn't mess with, like a shared database), but my point stands.

Use a linter

Save yourself from losing hours of your life to typos and hard-to-spot errors (IE trailing comma bug, anybody?) and let a linter find them for you.

I use Sublime Text with the Sublime-JSHint package to show lint errors as I type. (This also saves a lot of time I'd otherwise spend reloading the app I'm working on only to get a syntax error because I forgot a curly brace.) But you use whatever works for you and your editor of choice.

Running JSHint on an older codebase or one without consistent style conventions for the first time might make you want to cry, but persevere. You don't have to fix every single warning just because it's there. (I say this for my own benefit as much as yours, trust me. I also suggest making a very lenient .jshintrc that disables the rules that are more like style preferences and only checks the likely-to-cause-bugs ones.)

The first time JSHint finds a case statement that's falling through accidentally or a variable that's being unintentionally reused and causing mysterious problems, it will totally be worth it.

Learn your editor

As previously established in the first installment of this advice, I love nerding out over tools, and this is another instance of that. But learning how to move around a large codebase quickly is ridiculously helpful when debugging, and well worth the time spent learning some of the intricacies of your editor.

My editor of choice is Sublime Text 3 in Vintage mode (yay vim keybindings), and these are the commands I use all the time while debugging. I'm including the Sublime shortcuts for these since I know them, but most editors probably have similar features.

Fuzzy file matching Cmd-P
Look up a file based on a partial filepath. Saves so much time when you're looking for SomeWidgetView.js but you're not entirely sure where in the repo it lives.
Function search Cmd-P, then type @
List all of the functions in a file (and search them, if you start typing a function name after the @). A good way to get an overview of a giant file you're unfamiliar with.
Search within a file, jump to a line number / and gg<line> (specific to Vintage mode)
Become familiar enough with these that they're automatic, so that answering a question like, "Huh, where else does this function get used?" or looking at the line with a syntax error is frictionless.
Go to definition Opt-Cmd-↓
Jump right to the definition of a function. I've had mixed results as far as this actually working in ST, but it saves a lot of digging when it does.

Leave a path behind you

Once you've spent twenty minutes puzzling out exactly what data that getData() function retrieves and why, do future debuggers a favor and leave a comment documenting your findings. If making another person's task much easier in the future isn't motivating enough, consider that the person debugging next time is very likely to be you (especially now that you've become the dev who fixed a bug in this code most recently). Or that if you can't finish this task today, you might have to figure this out again when you come back to the code tomorrow.

If another dev gives you a helpful explanation of how a library works or lets you know about a quirk of the framework on IRC or via email, take a minute to clean it up and post it on the dev wiki or wherever other devs can access it. It doesn't require a lot of time and it is definitely worth it.

Run your own mini Heroku with Dokku

Sun, 09 Mar 2014 16:00:00 GMT

I find building toy projects much more fun if I can get them on the internet to show them off to people. Having things that only run on my local machine just isn't the same. When you're using technology newer or more complicated than the LAMP stack, though, deploying things to a server can be a challenging (at least if your patience for being a sysadmin is as limited as mine).

Heroku is a popular choice for painlessly getting small projects online, but I'm difficult and the 30 second start up time for rarely used sites on the free tier annoys me, and I'm not ready to pay $30 or so a month to host a toy project.

During my Hacker School batch I got my multiuser drawing app running on a $5/month Digital Ocean droplet, but configuring Nginx to work with Node and websockets pushed the limits of my sysadmin skills and patience.

However, Digital Ocean offers many different preconfigured server images, and I recent set up a droplet with their Dokku image. It has turned out to be perfect for getting small projects online without too much work and I love it.

Dokku is basically a tiny Heroku clone written in bash. It uses Heroku buildpacks to compile projects, so if your application runs on Heroku it should run on Dokku. Once your Dokku server is running, adding and deploying a new project is as easy as doing this:

git remote add mywebsite dokku@mywebsite.com:myproject
git push mywebsite master

Once everything finishes building, your project will be running at myproject.mywebsite.com!

If this seems appealing, there are a bunch of good tutorials for getting started, and the Dokku docs are also helpful. This is what I did, and the resources I used:

Set up a subdomain with wildcard DNS
I had to work around my hosting a bit for this: I was using NearlyFreeSpeech, where this blog is hosted, for DNS, but they don't support wildcard subdomains. However, Gandi, where I registered my goofy domain hack address in the first place, does. After switching my DNS over, I was able to point alliejon.es here, while by.alliejon.es (and *.by.alliejon.es) point to my Digital Ocean server.
Create a Digital Ocean droplet using their Dokku tutorial
(DO has a lot of really great beginner articles on server setup besides this one that are worth looking through, too.)
Deploy my own application
I had tested the Dokku setup with Heroku's sample node application (as suggested by the tutorial above), and that worked great, but my first attempt at deploying drawtogether just led to mysterious 500 errors. So, a tip: don't forget to add a Procfile to your project repository so Dokku actually knows how to run your application. Once I figured that out, everything worked perfectly with no additional configuration on my part. Awesome!

All told, even with my DNS fiddling, a lot of googling because I was wondering if websockets would work out of the box (they do) and forgetting to create a Procfile, it only took me a few hours to get everything running. Definitely worth it for simple deployment with git and inexpensive project hosting.

Seven Databases in Seven Weeks: Installing Erlang and Riak 1.0.2 on OSX 10.9

Fri, 14 Mar 2014 16:00:00 GMT

I've been reading and working through Seven Databases in Seven Weeks lately, but when I got to the chapter on Riak this week I ran into a problem: the latest version of Riak (1.4.8 as of right now) is different enough that I couldn't follow the book's tutorial, and installing the older version of Riak used to write the examples turned out to be a little bit of a challenge.

Since I did figure it out eventually, here is an explanation of how I got Riak 1.0.2 running on my Mavericks machine. The main issue (as far as I can tell, though I am far from an expert on this stuff) is that these older versions of Erlang and Riak don't compile correctly with clang, which is the default C++ compiler for OSX 10.9. So you have to install gcc instead, and make sure Erlang and Riak are built using that.

To start, I installed gcc via homebrew: brew install apple-gcc42.

Then I basically followed Basho's instructions for installing Erlang via kerl. I did have to change a few things, so I've documented the whole process in this post, but most of this came from their docs.

First install kerl, which is basically like virtualenv or rvm for Erlang.

curl -O https://raw.github.com/spawngrid/kerl/master/kerl
chmod a+x kerl

Then create a ~/.kerlrc file containing this configuration, so that Erlang will be compiled using gcc instead of clang (the rest of the build settings are as suggested by Basho's docs).

KERL_CONFIGURE_OPTIONS="CC=gcc-4.2 --disable-hipe --enable-smp-support
                        --enable-threads --enable-kernel-poll
                        --enable-darwin-64bit"

Now you can tell kerl to build, install and activate Erlang. Version R14B03 is the latest one that will work with Riak 1.0.2. (FYI, this version of Riak is not compatible with Erlang R14B04, which I learned the tedious way.)

./kerl build R14B03 r14b03
./kerl install r14b03 ~/erlang/r14b03
. ~/erlang/r14b03/activate

With Erlang installed, you're ready to set up Riak. You can download the Riak 1.0.2 source from Github. Riak won't build correctly with clang either, so you need to specify gcc when compiling again. (I'm not sure that running make rel is neccessary here, but that's what I did.)

CC=gcc-4.2 make rel
CC=gcc-4.2 make devrel

And now (whew!) you should have a functional old version of Riak, and you can get back to learning about databases.

Adding syntax highlighting to knitting patterns

Sun, 23 Mar 2014 16:00:00 GMT

A few weeks ago a coworker sent me Karen Shoop's awesome essay about the similarities between knitting patterns and regular expressions. While thinking about that, as well the as the knitting chart generator I'm going to make someday, really, it occurred to me that syntax highlighting knitting patterns could be genuinely useful. So I tried it!

If you're not familiar with knitting patterns, they use a semi-standardized set of symbols and shorthand to record exactly how to recreate a particular texture or garment. The pattern will tell you things like how many stitches you need, what sort of stich to make and when and how to add or remove stitches.

For example, here's the top of a sock I'm working on. The top of the cuff looks different than the rest, which is the result of a different pattern of stitches. (There are two kinds of stitches in knitting, knit and purl, though they are really the same stitch flipped 180 degrees—the back of a knit stitch looks like a purl stitch, and vice versa.)

The knitting pattern abbreviation for the cuff (at the top of the photo) is this:

k1 p1 until end of round

That translates to "knit one stitch, then purl one stitch, then knit, then purl ... until you get back to where you started". (This sock is being knit as a tube, hence "round".)

Instructions for the lower part of the sock are like this:

k3 p1 until end of round

Almost the same, but with a different ratio of knits and purls, which makes a different texture.

For another example, here's a closeup of a hat I made. Same two stitches, but in more complicated patterns:

A section of that hat's texture (the diagonal stripes) would be transcribed like this:

k2, p2 to end of round.
k1, (p2, k2) to 3 sts before end of round, p2, k1
(p2, k2) to end of round
p1, (k2, p2) to 3 sts before end of round, k2, p1

So you can see how that starts to get more challenging to read and follow. (The parentheses indicate a series of stitches you should repeat.)

Wikipedia has an interesting taxonomy of the kinds of knitting abbreviations, but that didn't seem like the most practical way to highlight the different portions of knitting pattern instructions. Instead I thought about the things that most often cause me to make an error while following a pattern, the parts that should be more distinct from each other while reading. I came up with this list:

which kind of stitch to make (knit or purl)
portions of the pattern that repeat
adding or removing stitches

I wasn't really interested in writing my own syntax highlighter (yet, anyway), so I used Prism.js and wrote my own knitting pattern language definition, giving each of those pattern features their own distinct color. (And, in a neat circling back around to the essay I linked to at the beginning of this post, I used regular expressions to do so!)

Here's the more complicated pattern from above, this time with syntax highlighting:

<pre><code class="language-knitting">k2, p2 to end of round. k1, (p2, k2) to 3 sts before end of round, p2, k1 (p2, k2) to end of round p1, (k2, p2) to 3 sts before end of rnd, k2, p1 </code></pre>

Much easier to read, I think!

For another, more complicated, example, here's a portion of this lace tank top pattern. This pattern includes increasing and decreasing stitches (the abbreviations in teal and yellow), and repeated instructions (in orange).

<pre><code class="language-knitting">k2, p2 to end of round. Rnd 1: K1, *k4, sl1, k1, sl1, k5; rep from * to last st, k1. Rnd 2: K1, *k3, k2tog, yo, k1, yo, ssk, k4; rep from * to last st, k1. Rnd 3: K1, *(k3, sl1) twice, k4; rep from * to last st, k1. Rnd 4: K1, *k2, k2tog, yo, k3, yo, ssk, k3; rep from * to last st, k1. Rnd 5: K1, *k2, sl1, k5, sl1, k3; rep from * to last st, k1. Rnd 6: K1, *k1, k2tog, yo, k5, yo, ssk, k2; rep from * to last st, k1. Rnd 7: K1, *(k1, sl1, k2, sl1) twice, k2; rep from * to last st, k1. Rnd 8: K1, *k2tog, yo, k1, yo, ssk, k1; rep from * to last st, k1. Rnd 9: K1, *sl1, k3, k1tbl, sl1, k4, sl1, k1; rep from * to last st, k1. Rnd 10: K2tog, *yo, k3, yo, sl1, k2tog, psso; rep from * to last 6 sts, yo, k3, yo, ssk, k1. Rnds 11-12: Knit. </code></pre>

I've put my (pretty simple) language definition up on Github, if you're interested in trying it out. I'm sure there is room for improvement in the regexes I've written, though this was definitely a great way to get practice with writing more complex ones than I typically do.

"Girly" games, programming, and the fractal gender binary

Sun, 30 Mar 2014 16:00:00 GMT

Last week GOG had the Creatures series of video games on sale, and I couldn't resist getting a copy that would run on my current computer and spending an evening immersed in gaming nostalgia.

As a kid and a teenager, I was fortuate enough to have easy access to the family computer (and later my own machine), and I spent many happy hours raising imaginary animals, figuring out how to access the alt.games.creatures newsgroup, downloading game add-ons and trying to build my own.

I was similarly fascinated with the Petz games, and loved breeding strange hybrid animals and trying my hand at editing the files that defined the appearance of the cats and dogs in the game. (If you want to see some amazing examples of the game being pushed to its limits, this tumblr is hilarious.) I learned how to build websites in part so I could share the things I had made.

Then I discovered MU*s, which, besides being ridiculously fun, greatly improved my writing ability and introduced me to friends I still have, over a decade later.

Looking back, I am amazed at all of the technological skills I learned in the context of playing these games I loved, that I would otherwise have been too intimidated to try. Editing Petz breeds required using a hex editor and reading documentation from others who had reverse-engineered the file format. MU*s had their own Lisp-like and Forth-based programming languages that I learned to use, as well as many admin commands inspired by UNIX. (Not that I realized that, at the time. The first time I used chmod on a web server many years later I was startled that it was familiar!)

What also amazes me is that I did all these things, and yet never really considered myself the kind of person who could be a programmer. I was nerdy, sure, but programming was this lofty skill that I was sure was probably beyond me.

It is hard to say now exactly why I felt that way (and I certainly don't remember ever being explicity discouraged from liking technology or having a career in it), but it seems likely to me in retrospect that I knew these were "girly" games, and my activities were not things "real" people who were into technology would do. People who became programmers wrote Quake mods, not custom breedfiles so they could have horses in their virtual pet game.

I think this Sociological Images post by Lisa Wade on the fractal nature of the gender binary provides an interesting framework for thinking about this[^1]. Playing games and creating your own add-ons for them is stereotyped as a masculine activity. But within that category there are more and less feminine games. First person shooters? Masculine (and, thus, cooler). Virtual pets? Feminine (and so not as cool). Even text-based RPGs, a very specific and tiny subcategory of games, have gender stereotypes: MUDs (which have stats and combat systems) are for dudes, while MUCKs (which are largely for free-form roleplay) have many more female players (or so everyone assumed when I played them, I'm not sure if it was actually true).

Even now that I am a programmer (and let's pretend I didn't feel a tiny bit hesitant saying that), this fractal nature of the gendering of things---and so also their relative value---reappears. Front-end web development seems to attract a greater proportion of women (though I admit I can't find hard data on this), and I don't think it is a coincidence it is considered "easier" than other programming specialties.

I don't really have a neat conclusion to these thoughts, except that gender and the ways we categorize the world are complicated, and we still have a lot of work to do to make everyone feel welcome in technology (and everywhere else).

[^1]: Just to be clear, I think that assuming gender is binary sucks (besides being, oh, totally inaccurate), and I wish things weren't socially classified this way. But these assumptions definitely exist.

Adventures in knitting machine hacking

Tue, 02 Dec 2014 16:00:00 GMT

My love of combining technology and textiles is well-documented already, so it is likely unsurprising I've had my eye on knitting machine hacks for a while.

Knitting machines are strange beasts. The most common ones, the machines I am familiar with, were made by Brother in the 70s and 80s for home knitters and are no longer manufactured. In college I took two semesters of knitting classes[^1] that used them, and they are odd and finicky and it isn't surprising they never quite caught on. They make fabric much faster than hand-knitting, but they also require a lot of skill to operate and they really aren't anything close to automatic.

You can still do a lot of neat things with them, though, and they are a much more accessible textile-making tool than many of the others I learned to use. (Maybe someday I'll have my own loom for weaving, but that seems pretty unlikely to happen while I live in a New York apartment.)

As far as hacks go, Becky Stern's is probably the most well-known, but I found this disclaimer pretty intimidating[^2] (and I can use knitting machines and Python already!):

Working with these machines is very difficult. Before you begin, look at your life, and what choices brought you to this point. Are you proficient at using the knitting machine's normal functions? Can you read and run Python scripts? Have you ever done any serial communication between your computer and another device before? If you answered "no" to any of these questions, work on these skills first before attempting to follow this guide.

I don't have any hardware skills, and tackling a modification to an expensive knitting machine as a first project seems likely to end poorly.

Happily, some enterprising German hackers made an alternative knitting machine hack that seemed more approachable to my intimidated-by-hardware self. All Yarns Are Beautiful designed an Arduino shield that interfaces with electronic Brother machines. Even better, the process is nondestructive and reversible, and it supports a less expensive model than the 930e used in the Adafruit hack.

A few weeks ago I finally decided I wanted to make my own hacked machine, despite my fear I would buy all these parts and ultimately get nowhere. I got lucky and found a Brother 910 with a broken card reader on EBay. The 910 originally read patterns off of mylar cards, which meant the pattern functions were unusable, but made it perfect for my purposes---the AYAB Arduino replaces that component completely.

This week the machine and the shield arrived, and I've made a lot of progress! I've been keeping notes and will be following up this post with others about the whole process. (And, hopefully, some finished products!)

If you'd like to see a Brother 910 in action, I found this cool video of Karen Allen demonstrating how she uses one to make sweaters. (Though she's using the original card reader for patterns, she's done some knit hacking too! I've never seen someone use two carriages at once to knit with that many colors.)

[^1]: How cool is it you can study this in college? I still think it's amazing. (Probably moreso now that I am not fighting with knitting machines at 2am though, hah.) [^2]: Having now attempted this project, I will say the disclaimer isn't wrong exactly, but I think you can do fine with two skills out of three. I would say one of those skills should be using a knitting machine, though, since they can be tricky enough on their own without the complication of a hack.

Knitting machine hacking: flashing the Arduino firmware

Sun, 25 Jan 2015 16:00:00 GMT

Wondering what this is about? See the introduction to my knitting machine hacking adventures.

Successfully flashing the firmware on my Arduino so I could use my AYAB shield took a bit of fiddling, especially because I'm running OSX instead of Linux.

I'll list the straightforward (as I can manage) steps that actually worked first, for anyone else trying to do this, and the messy narrative version afterwards (since that's the fun/infuriating part!).

If you use these instructions and run into problems, or have suggestions for improvements, please get in touch! I'm happy to help though I'm far from expert in these things (so while I hope these instructions won't mess up your Arduino, I definitely can't promise that). This is what worked for me, though.

How to install AYAB Arduino firmware on OSX

AYAB comes with a Python-based GUI, but I couldn't get it to run successfully on my OSX machine, and it isn't neccessary for flashing the Arduino firmware anyway. These instructions explain how to get the firmware image and use avrdude to flash the firmware from the command line.

I did this on OSX Yosemite (10.10) but my guess is it would work fine on any relatively recent version of OSX.

Install avrdude 5.x (I used 5.11.1 specifically)
Homebrew only has a formula for avrdude 6.x, but if you don't want to build avrdude yourself, an older version is downloadable from Objective Development as part of their CrossPack toolkit. You need the 2012-11-28 release. Here is a direct link to the dmg file.
Download or clone the AYAB software repo
Find and edit avrdude.conf or download a corrected version The avrdude config provided by AYAB won't work when used on a computer without parallel ports (which I'm pretty sure means any modern Mac).

I put the modified version I used on Github if you'd like to download it.

Alternately, to edit it yourself, find the config file in the repo at /plugins/ayab_plugin/firmware/avrdude.conf. Do a search and replace on this file, replacing any instances of par; with serbb;. (Thanks to this Arduino forum post for this fix.)
Find firmware.hex in the AYAB repo
The path relative to the AYAB repo root directory is /software/python/ayab/plugins/ayab_plugin/firmware/[KNITTING MACHINE MODEL]/[ARDUINO TYPE]/firmware.hex. You'll need to pick the directories that match your knitting machine and Arudino. There are different files for the different combinations.
Determine which USB port your Arduino is connected to Your Arduino needs to be connected to your computer via the USB cable for this command to work (and for the firmware flashing you will be doing shortly!). Running ls /dev/tty.usbmodem* should show you the correct serial port. (I think this is dependent on your OS version and Arduino model, so you may have to do a bit of Googling if this specific command doesn't work.)
Actually flash your firmware!
Now you should have collected all of the information you'll need. Here's the avrdude command to run: avrdude -v -p atmega328p -C "[PATH TO AVRDUDE.CONF]" -c arduino -P [ARDUINO SERIAL PORT] -b115200 -D -Uflash:w:"[PATH TO FIRMWARE.HEX]":i A note: I have an Arduino Uno, and you may have to change the -p atmega328p too if you're using an Arduino Mega, but I'm not sure.

So if all goes well, avrdude should do its thing and flash the firmware, and you're ready to install your AYAB shield, yay!

How I figured all this out

In short: trial and error and a lot of Googling. (As with so many things!)

My previous Arduino experiments have been at the "making an LED blink" level, just to give you an idea of my familiarity with hardware things (and to explain all my fumbling around).

I started by trying to get the AYAB GUI to run on OSX, since it has a firmware flashing utility built in, but after an hour or so of fiddling with Python dependencies and trying to figure out how to install what it was was missing, I decided this wasn't going to be a very productive direction.

Instead, I set up a Linux VM using Parallels (though I bet you could use VirtualBox for this too) and installed the GUI in that. This was much less painful and following the AYAB-provided instructions seemed to work.

This turned out to not be the case with the firmware flashing utility, however. The main part of the application, which you use to control the knitting machine once everything is set up, worked fine. But trying to run the firmware utility just gave me an error.

I'm not sure if using a virtual machine for firmware flashing would have worked, because I never got that far. Using pip to install AYAB (as per the instructions) doesn't seem to install the avrdude utility the GUI runs behind the scenes, so it fails to do anything.

I figured this out from error messages in the debugging console that opens with the GUI, once I realized the console was actually providing me with useful information.

avrdude does seem to be in the repo, so I spent a while trying to manually copy it to the right directory so that the AYAB GUI could find it, but for whatever reason I couldn't get this to work. While I was trying this, though, I realized that the console error messages also showed me exactly what the GUI was trying to do for me, which was run avrdude with a bunch of arguments.

A little bit of research into what avrdude does (since I'd never heard of it before) confirmed that running it was probably all I needed to do, and showed me what files from the AYAB repo I needed. (The command the GUI runs is the one I provide above in the step-by-step directions.)

And so I saved a copy of that command, and tried installing avrdude on OSX! (OSX because I'm not a very efficient Linux user, and I was worried the virtual machine stuff would mess things up.)

Homebrew's version of avrdude didn't like the config file from the AYAB repo, and then I realized that the avrdude version provided by AYAB was 5.x and Homebrew's was 6.x---that seemed likely to be source of the problem. Since the avrdude.conf file was huge and I wasn't entirely sure which parts were relevant or really what it was doing at all, I guessed it would be easier to try and find an older, compatible version rather than update the config for the new version.

After rummaging through a bunch of tutorials and search results and seeing if I could build an 5.x version of avrdude from source (nope) I eventually found the old version of CrossPack I linked to in my instructions.

Now I had a running, hopefully-compatible version of avrdude! So close! And yet still not quite there apparently!

Running avrdude with all the right (I hoped) arguments and file paths gave me the mysterious errors parallel port access not available in this configuration and programmer type not specified. Googling those errors turned up this forum post which seemed to describe exactly the problem I was having (and my laptop certainly doesn't have a parallel port), so I tried the suggested solution.

And with that last change, it worked! avrdude ran for a minute or two and then I had a newly-flashed Arduino.

In retrospect, it is hilarious that I thought "just" buying an Arduino shield for my knitting machine might be too simple a project, and I wouldn't feel very accomplished getting it to work. Definitely not the case! I felt pretty damn victorious by the time I was done with this part of the process.

Next up: installing the Arduino in the knitting machine and the moment of truth!

Knitting machine hacking: putting it all together

Fri, 29 May 2015 16:00:00 GMT

Previous knitting machine hacking posts: introduction, flashing the Arduino firmware

Thankfully, after the complicated process I went through getting my Arduino set up to work with the AYAB shield, actually connecting everything to the knitting machine was straightforward!

The video from AYAB showing how to open up your knitting machine and connect your Arduino is great and very clear. I had to snip a cable tie to get enough slack in some of the cables and remove a plastic clip from one of the connectors to get it to fit when connected to the shield, but it all worked out fine.

Since my knitting machine has a broken card reader anyway, I didn't bother keeping it. I did keep the plastic outer case, but removed the electronics and display, which were just attached with a few screws. The empty space created is big enough to fit the Arduino and shield (though I have to disconnect some cables from the Arduino), which makes it much easier to store everything.

I was dying to find out if this was all going to work and I couldn't find documentation for the command line control utility (though I've since figured it out), so I used the AYAB GUI running in an Ubuntu virtual machine[^1].

It took a bit of experimentation to get the knitting process initialized correctly, but it totally worked! I was ridiculously excited at this point, as you might imagine. (And yes, I did need a new sponge bar for my machine, which is very often the case with used knitting machines. Thankfully I had a spare!)

I did a lot of testing and experimenting without yarn as I was getting the hang of using the AYAB GUI. You don't need to have yarn in the machine to test the needle controls---you can run the empty carriage back and forth to try out the process first.

Two things to note about the GUI that I figured out through trial and error: it only supports PNGs (as far as I can tell? JPGs are included in the example files, but they did't work for me), and the file width needs to be smaller than the number of needles you've selected to work with.

It works! Actual knitting!

Here it is, video evidence of my newly-hacked knitting machine in action. (Also including a cameo from my cat Abe, who loves all things yarn-y and was very curious about this strange noisy machine.)

In case you couldn't quite see what I knitted at the end of the video, here's the source image (scaled up significantly; there is one pixel for each stitch so the original is 50 pixels wide):

And the finished product:

What's next

Since I wrote most of this post (in February, whoops) I've acquired a ribber for my knitting machine, which is a second set of needles that creates a lot more possibilities for knitting images into fabric.

I'd also really like to write my own GUI for my machine, probably in Node with a web interface. I have some UI ideas brewing ...

[^1]: This worked fine using Parallels on my Mac; the only configuration neccessary is to tell Parallels to "pass through" the Arduino's USB port to the VM from the Devices menu.

Smoother remote live demos with OBS

Fri, 30 Oct 2020 16:00:00 GMT

Like the rest of the world, I've been doing a lot more communication over video chat in 2020. I've had remote teammates for a long time, so not much has changed with how I communicate with coworkers. My volunteering, however, feels totally different: for the past few years I have helped teach web development to high school students through CodeNation, and this is the first school year the class has been entirely over video rather than in-person in a classroom.

Teaching effectively requires a lot of skill, and teaching remotely feels even more challenging! (I hope this isn't news to anyone.) Technology can't fix most of the challenges, but I still couldn't resist experimenting with my setup to make presenting and coding remotely a little easier. I went into this thinking I was just indulging myself in some fun yak-shaving, but I was pleasantly surprised at how little fuss a better setup required, so I'm documenting it here in case other folks find it useful! (And though I've been using it for teaching so far, this may end up upping my work presentation standards too ...)

My goal was to make it easy to switch between slides in presentation mode and a code editor (Glitch in the browser, in my case, but this would work just as well for a standalone program) without having to share my entire screen. This means I can keep a class schedule, presenter notes, the meeting chat and whatever else visible on my own screen without showing them to everyone. I'm not dealing with audio setup here because it's not neccessary in Zoom; you can continue to use your regular webcam and audio.

OBS (yes, the same program Twitch streamers use) makes this pretty easy! And while I haven't even scratched the surface of all the fancy things you can do with OBS, once you have this set up it should be straightforward to experiment with fancier things. I'm assuming you're using OSX and presenting over Zoom in these instructions, but the general approach should work for other OSes and conferencing software too.

There are only a few setup steps:

Install OBS and a virtual camera plugin
Configure your "scenes" (the different sections of your screen you'd like to display)
Share the virtual camera in Zoom
Present!

Installing OBS

Download OBS from the project's website and install it.

OBS for Windows comes with a virtual camera plugin pre-installed, but you need to download a separate plugin for OSX. I used obs-mac-virtualcam. Download the latest .pkg file from the releases page and install it as well.

When you start OBS for the first time, it will ask you about what preset settings you'd like. You can cancel out of this window, it doesn't really matter for this setup.

Set up your scenes

OBS lets you combine a bunch of different input sources into one video output. For this simple setup (switching between two application windows) you only need to use "window capture", but it's worth experimenting with some of the other options! You can display the output of a specific web URL or add a nice background image or webcam feed pretty easily.

To add a new scene:

Make sure the "Scenes" tab is selected (you should see this in the lower left of the OBS interface by default).
Click the "+" button and name your scene something that makes sense to you (I usually pick something like "Slides" and "Browser").
Then click the "Sources" tab, click "+" again and select "Window Capture" from the dropdown.
Choose "Create new" and then, in the "Properties" dialog that opens, choose the window you want for your first scene.
If you can't tell windows apart from their titles, selecting a window will show you a preview. If you just see a list of windows named "void" or it seems like things are missing, you may need to grant OBS access to screen recording.
Click "OK".

In the main OBS window, you can now manually drag and resize the window capture to make it a custom size. If you just want this one window, I find it easier to right click on the preview and select "Tranform > Fit to screen" to automatically make the whole capture visible and as big as possible.

For your second scene (and any additional scenes), repeat these steps to create a new scene and a new window capture.

Use the virtual camera in Zoom

To connect OBS to your meeting software, you can use a "virtual camera". This feature needs to be turned on in OBS: go to the "Tools" menu and select "Start Virtual Camera". (If it says "Stop Virtual Camera" the camera is already on, and you don't need to do anything. If you don't see either option, double-check that you've installed the virtual camera plugin.)

You don't need to do anything else in OBS! Don't worry about the "Start Streaming" or "Start Recording" buttons; you just need the virtual camera on to send OBS output to Zoom.

Zoom works particularly well with this setup because you can share the output of a second camera while keeping your regular webcam unchanged. Zoom provides more detailed instructions if you'd like them, but the short version is: in the dialog that opens when you click the "Share Screen" button in a meeting, select "Advanced" and then "Content from 2nd Camera". Click the "Switch camera" button to the virtual camera output if you don't see it right away.

Present!

You should be good to go switching between your different scenes! As you're presenting, select the scene/window you'd like to share from the Sources tab in the OBS window and it will update the camera output.

I experimented with using the Advanced Scene Switcher plugin to switch between scenes when I moused over the window associated with each scene, but in the end that felt too fussy (and despite moving my mouse very carefully it was still easy to accidentally switch scenes when I didn't intend to). I think OBS also has a lot of other built-in automatic scene switching options you could potentially use, but for this not-very-complicated setup manual switching seems simplest.

Building Javascript SSR for Etsy.com

Thu, 04 Nov 2021 16:00:00 GMT

In 2021 I architected and then tech led the implementation of Etsy's server-side rendered Javascript infrastructure. We used Preact and the islands architecture, which let us create a smooth developer experience without sacrificing our frontend performance.

I wrote about it in detail on the Etsy Engineering blog Code as Craft.

Modernizing legacy codebases with Marianne Belotti

Tue, 08 Nov 2022 16:00:00 GMT

As part of Etsy's Code as Craft talk series, I spoke with Marianne Belotti about our experiences wrangling with legacy code. It was a really fun conversation, and I'd highly recommend her book Kill It with Fire.