Search

I Have Books

Master Space and Time With JavaScript


Download Book 1 For Free


Or…


Buy the rest of the book


Rails Test Prescriptions

B&N Affiliate Link
Pragmatic Link

On The Blog

Blog Index
The journal that this archive was targeting has been deleted. Please update your configuration.
Navigation

Rails Test Prescriptions Blog

Friday
Jun142013

State of The MSTWJS Union, June 2013

Here is the state of the Master Space and Time With JavaScript Union as a I currently understand it.

First, you can, you know, buy the book. Please.

An update to the Ember book should be going out next Monday or so. It has about 10-15 page chapter walking through a routing example with outlets and nested routes and the like. That’s puts the Ember PDF at just over 90 pages (including the back matter and contents and stuff), which compares favorably with the lengths of the other three books (97, 97, and 120, I think).

I think there are still a few things missing from the Ember book.

  • More on views and handlebar helpers
  • Ember data details
  • Ember-testing details
  • Anything else the Ember team adds before 1.0 (async routing, for example)
  • I’d like to deal with some real-world examples, at least briefly, of things like authentication

This points to a final page count roughly in the range of the backbone book, about 120 pages or so. Actually, probably more. Sheesh.

What’s tricky about this is that ember-data and ember-testing are still not quite done enough to write about them without risking throwing everything out and starting over, so it’s hard to give a final schedule.

The other books are quiet at the moment, though the Backbone book could stand an edit.

Thinking Ahead

Meantime, there’s a slow but steady trickle of requests for a MSTWJS Book 5: Angular. (A five part trilogy would make me feel like Douglas Adams)…

I’m genuinely not sure if I’m going to do this. The considerations are:

Opportunity. There’s clearly some space here for a book that really explains Angular. According to a Twitter poll — which I’m sure is totally reliable — the overwhelming majority of existing MSTWJS customers would expect to pay for a Book 5. This actually goes a long way toward making me think this is worth my time. Writing tech books is kind of a hobby for me, but I’d still like to feel that I can make a little bit of money from them.

On the other hand, I’ve been working on this book for two years now. That’s a long time in Internet years, and doing an Angular version probably means another three to six months of work. There are other things I want to write, and it wouldn’t kill me to blog regularly again. (Actually, that implies that I once blogged regularly, which is not really true.) I really want to do this project book, and I’m starting to feel like I have new things to say about testing.

Anyway, right now I’m thinking that I’ll at least play with Angular a bit to see if I have any affinity for the tool at all. If I hate working with it, I’m not writing about it. If it passes that hurdle, then we’ll see.

If I do it, the likely scenario would be — don’t hold me to this, I’m just thinking out loud: * Priced at $7 or $9 based on length * Existing 4 book bundle owners would get a discount, possibly a limited time discount depending on what I can do via DPD. * At that point, new customers would get a five book bundle. I might raise all prices at that point — the hypothetical five book bundle would be nearly 600 pages. Not sure what that pricing looks like yet. $9 per book with a $25 bundle is one possibility.

Please let me know if you have thoughts about this — my perception of people’s willingness to buy an Angular book in this series is a major factor in whether I do it.

And, oh yeah, buy the book.

Thanks.

Saturday
Apr202013

Status Update

It’s been well over a month since the last official update to Master Space and Time with JavaScript, and since I was hoping to get updates out more-or-less weekly, it’s probably a good idea to check in and let you know what’s going on. (Could be worse, though, I’m still hoping to post my top 10 books I read list. From 2011.)

All of the MSTWJS customers out there have been either very patient in waiting for the next update, or you are completely disengaged. Personally, I’m choosing to assume patient.

I am continuing to work on the book, it’s just slowed down quite a bit. The next update will most likely be sometime after RailsConf — ideally sometime in the week or so following. So by, say, May 10th. I post this date publicly to increase the chance that I’ll actually hit it.

There are a few reasons why the book’s progress has slowed.

To some extent, it’s a deliberate slowdown so I don’t have to rewrite the thing a dozen times. Yes, Ember is in an API freeze, but they are still adding new things that preserve compatibility, and Ember-data is decidedly not in an API freeze. Just in the last week or so, integration testing tools are starting to emerge — see this discussion for details. Honestly, the fact that I had to throw my hands up over integration testing in the last update was very disappointing, and I’d very much like to get that working in the example.

There’s also been some fumbling about what I want to cover in the rest of the book and how I want to get there. This one, I think I’m getting a handle on.

I also got busy. For example, I’ve been starting this weekly sort-of-screecast series for Table XI called XI to eye. I’ve never done anything like it, but I’m pleased with how this is going. There are five so far, you can see them all at http://www.tablexi.com/blog/category/xi-to-eye. Please do check them out.

I’m also doing two sessions at RailsConf — a normal session comparing rich client MVC with the 37Signals Basecamp approach and an intro track session on testing complex systems. More on those next week, but I’m excited for both of them.

There’s also been some laziness, and some lack of momentum caused by the combination of the previous points.

Still, hoping that this will move a little more rapidly in May — we’re now coming on the two-year anniversary of me starting the project (though I suppose I’d be done if I hadn’t decided to pick up Ember as a topic), and I’m certainly ready to move on. (I have some idea what my upcoming writing projects will be, just not sure which one I want to tackle next.

Thanks for your patience, if you are enjoying the book more will be coming — there will also be another errata catch up on the first three books. Please do help spread the word, or maybe buy it yourself.

Monday
Jan212013

Announcing Ember! Master Space and Time With JavaScript Book 4

You have no idea how happy I am to announce that Book 4 of Master Space and Time with JavaScript: Ember is now on sale. You can buy it at http://www.noelrappin.com/mstwjs

It’s not done, of course. But it’s off to a good start, and I think it’s going to be great. Here’s the state of the world:

  • Release 1 of the Ember book is $7, just like its recent siblings. The four book bundle is still $15.

  • Release 1 is about 20 pages of content (plus the same intro, acknowledgments, and so on). It covers setting up the Time Travel app with an Ember front end, then the basic concepts involved in getting data from the server, displaying it to the browser and responding to actions. We basically end at a chapter break, so it’s a contained example, though obviously the rest of the book will take it farther.

  • The code is written against Ember 1.0pre4, which means its current as of yesterday.

  • Unlike some of the other betas, this one has been reviewed by a few people, including people who know more about Ember than I do, so I’m pretty happy with how it’s going so far.

  • We don’t cover testing yet (in part because the framework does so much that there isn’t much logic to test yet). It’s coming, though. Trust me.

  • The plan is for basically weekly releases until done.

  • I don’t know how long this is going to get — to some extent it’s going to depend on the reception, more sales = more encouragement to write. It will be at least in the 100is page range as the previous books. If it gets much longer than that, I may raise the price at that time.

That’s the scoop. Buy it: http://www.noelrappin.com/mstwjs, enjoy it, spread the word.

Thanks

Monday
Jan072013

State of My Stuff, January, 2013

The state of the Noel Rappin publishing — I can use the word “empire”, right?

MSTWJS 3: Backbone

Master Space and Time With JavaScript, Book 3: Backbone is draft complete as of today’s release except for a page or so of summary at the end, which I will probably wait to write until the Ember chapter is complete.

The next release will be an edit pass, to clean up typos and bad sentences, smooth over the explanation, fix errata and generally polish stuff. I haven’t done any of this on the book yet, so it’s possible there will be some bigish changes depending on what I think is going on.

Any further releases will be errata and cleanup based.

MSTWJS 4: Ember

I’m starting to get actual questions about the Ember version.

In the wake of the Ember team getting their newest router API into master, and also updating a lot of their documentation, I’ve broken ground on this one and am moving forward.

I will release this in a pretty early state, I imagine, and continue with the weekly updates until it’s through, which I expect will be on the order of ten weeks. Ish.

I have two criteria for the initial release. The first is that I need the example in the book to go far enough to display some things on the screen and show off some event and property processing. Secondly, I have a couple of people who have volunteered to sanity check the book, and I’d like at least one of them to sign off on it before anything insane I write is exposed to the general public. So, optimistically, Jan 14, but maybe a week or two after.

MSTWJS 1 and 2

I do have some standing errata that have been reporting. I would expect not to see an actual update until after the Ember book is on sale, not least because then the update email can be used to tell all the free book one customers that the Ember book exists.

Next Projects

So, I do know what my next project is going to be, I’m planning on self publishing it. It’ll be Ruby and testing based, but I don’t plan to start on it until after the Ember book is out in the wild. I came up with an outline while I was waiting for the Ember team, but now that I they look solid, I want to get that out first.

Thursday
Dec272012

More Lessons Learned

Last year, when Obtiva was purchased by Groupon, I wrote a "what I learned" post talking about things I thought I came to understand about software projects after working on a bunch of them. Now that I've moved on from Groupon, I started to think about what, if anything, I learned while I was there.

I keep coming back to three different things -- this is a more personal set of lessons than the last batch, so maybe they'll be less generally useful. Maybe not, though. Think of it as a little long-term career management advice.

These are all things that I think I kind of knew going in, but there's knowing something and knowing something, know what I mean?

Also -- nothing here is meant to say anything against Groupon Engineering -- I freely admit that what I'm talking about are issues of me managing my life and time.

Coding is the Input to Everything I Do

I like coding. (I thought I'd start with something controversial…) I like it a lot. That said, I've never been the guy who needed hours more coding in his week above and beyond work. For most of my professional career, my personal projects have been writing projects. And while I love to continually learn new things, it's been a long time since I felt the need to forgo sleep to do so.

Groupon was the first time in my professional career where coding was not a primary responsibility -- my primary responsibility was organizing training. And I'll stipulate that I went into it eyes open. It sounded pretty good. No coding project deadlines, and I love to teach. A room of students who have to listen to me sounds good.

At first, I was actively building the curriculum, that was pretty good. Then there was less curriculum work to, and in for reasons that I don't really need to detail, the things asked of me became a lot more administrative. The point is that as I got further away from everyday coding, I felt like I got less good at all kinds of things that I want to be good at. I didn't have interesting problems to blog about, I wasn't learning new things as much. I felt less credibility as a teacher and speaker as I got a little removed from practice. Combine this with my occasional tendency toward impostor's syndrome, and things got less fun quickly.

There were options available to me at Groupon that I chose not to take for reasons good and bad. The main point for me is that by the time I realized what I was losing, it was hard for me to feel like I could get it back. The point for you, I guess, is to be confident in knowing what you like to do and what parts of your work are satisfying. Know what you need to have as your inputs to be successful over the long term.

Big Companies are not Small Companies.

I know, duh.

I've joked for years about "big company problems" vs. "small company problems". In a small company you have to maintain the CI server yourself. In a big company, there's a whole IT department, but it takes you six months to requisition a CI server. (True story, at Motorola. Also they told me the CI server was causing too much network traffic, and did I have to run it after every checkin.)

There are two structural issues at big companies that have tended to drive me batty. The first -- that people are often making decisions about other people who they don't know and only have a dim idea of what they do -- was not a major issue for me at Groupon. The second -- that big companies tend to encourage people to specialize -- decidedly was.

I want this post to be about me, not about them. So here's a related story: when I first entered the job market, I wound up with two serious offers, salary identical. One of them was at Nokia, for an R&D department in Boston where I would have done usability research tangentially related to what I had been doing as a grad student. The other was what we'd now call a small web consulting shop, 12 people or so separated between two cities. I'd never done any web programming. They had done only a little bit more (they thought of themselves as documentary filmmakers by trade). When I went for my interview, the CEO of the company was vacuuming the floor of their 2-person Boston office.

Obviously I went with the tiny company, which even as I type this sounds kind of insane. And while I'd love to say something self-serving about how I picked the job that scared me, I don't think that's true (both jobs scared me). I do think, though, that I was excited by the prospect of doing a lot of different things. Which I did, the job turned out to be a kind of immersion in the entire lifecycle of software projects in the way that pouring ice water on somebody's head is kind of a way to get their attention.

Ever since, I've been happiest when I've been able to do all kinds of different things on a regular basis. Big companies, of course, tend to specialize because they need to, and because they can. Once you have 200 developers, suddenly you can spare somebody to be full-time in charge of improving training. (Well, almost full-time…) Which sounded great, for a while, but then, see point #1. I flatter myself that I do a lot of things well, and whether that's true or not, I still want to try to do a lot of things.

Introversion and local maxima

This one is a little tricky and it's really a personal anti-pattern.

Look, I'm in introvert in the classic definition. Every time I've taken a Myers-Briggs test, I bury the needle for I and N. On a day to day basis, one thing this means is that, while I usually like my co-workers (and my Obtiva/Groupon collegues are an outstanding group of people), I'll often choose, say, eating at my desk over going up to a group in the lunchroom and joining up with a group of people.

On a related note, for most of my last six to nine months or so at Groupon I was a team of one. Then the person I reported to left, and I wasn't even reporting to anybody else in Chicago. I swear this is true -- I was literally sitting in a corner with nobody on my two orthognal sides. I'm saying I was a little isolated. I'm also saying that of course I could have handled it better. That's part of my point -- one thing I learned is that what seemed like the best thing to do on a day to day basis, ended up being isolating in the long term. I would be able to go large chunks of a day without interacting with co-workers. Even a staggering introvert like myself has limits.

What does it mean?

Dunno. Just being a self-indulgent blogger. I expect most of you to read this, roll your eyes and say "Duh."

I do know that I've spent most of my six weeks at Table XI coding and helping run a small web project. I know it feels great even when it gets weird. It feels like I'm using muscles that got a little rusty. (I'd have some technical blog posts for you, but I'm backed up with the book. Coming, I promise.

Table XI provides lunch in house every day, which makes it a lot easier to actually talk to co-workers. Which is good. (I realize this lunch thing sounds totally insane to a significant percentage of you. I'm okay with that.) During my first week, I had a meeting where we planned out what kinds of things would happen as part of my first few months. One of my cards was "do something new". I don't know exactly what yet, but it's important to me to keep moving forward.

Thanks for listening, I hope you will, to paraphrase Tom Lehrer, find this valuable in bizzarre set of circumstances some day.

Wednesday
Dec052012

Leprechauns and Unicorns of Software

Something like 25 years ago, Bill James wrote an essay asserting that one difference between smart and dumb baseball organizations was that dumb organizations behaved as thought Major League talent was normally distributed on a standard bell curve, and smart organizations knew that talent is not (it’s the far end of a bell curve).

Hold that thought.

So I just finished reading Laurent Bossavit’s self published book The Leprechauns of Software Engeneering, which I recommend quite highly. Bossavit looks at some of Software Engineering’s core pieces of received wisdom: that there is a 10x productivity difference between developers, that the cost of change rises exponentially over the course of a project. The kind of thing that all of us have heard and said dozens of times.

Bossavit does an interesting thing. He goes back to the original papers, sometimes wading through a maze of citations, to find the original source of these claims and to find out what empirical backing they may have. Turns out the answer is “basically none”. One by one, these claims are shown to be the result of small sample sizes, or no sample, or other methodological problems. They’ve become authoritative by force of repetition. (Which doesn’t mean they are wrong, just that we don’t know as much as we think we do.)

If you are like me, which is to say you are fascinated by the idea of empirical studies of software, but deeply skeptical of the practice, this book will take your head to some fun places.

The Bill James analogy, for example. What James is talking about is that in order to accurately value what you have, you need some idea of the context in which it occurs. In the baseball example, to know how to value a player who hits ten home runs (which we’ll pretend is average, for the sake of oversimplification), it’s helpful to have a good sense of how many players are out there who are capable of hitting twelve, or eight. If you erroneously assume that there are fewer players capable of hitting eight home runs then ten home runs, then some bad management decisions are in your future. Specifically, you’ll overvalue your ten home run player (or more likely, overpay for somebody else’s ten home run player, when your own eight home run player is a fraction of the cost.)

I’m wary of taking this analogy too far, not least because it doesn’t necessarily reflect well on my overeducated typing fingers. There are all kinds of reasons to think that the curved for web developers is different than for baseball players. We don’t have a good idea of what the distribution curve of productivity is for developers, even if we had a good idea of productivity is (we don’t) or a way of measuring it (ditto) or any idea of how individuals improve or decline based on teams (guess what). That said, I do not think that I have been on an actual team where people were genuinely 10x better than other people. (Total newbies notwithstanding, I guess). Ten times is a lot of times, that’s one person’s week being another person’s half-day. Sustainably.

But see what I did there? I palmed a card. I said that newbies don’t count in my personal recollection of my teams productivity. Why not? For a good reason — I don’t think the productivity of somebody in intense learning mode has a lot to tell me about how software teams work. But that’s my decision, and it’s subjective, and suddenly I’m deciding which of my hypothetical data “really counts” and which doesn’t. That’s a normal process, of course, but it’s not How Science Is Supposed To Work. In reality, I’m already skeptical of the 10x finding, and pulling newbies out moves the data in a way I’m comfortable with, so I’m not likely to examine that decision too closely. (See Bias, Confirmation.)

I spent about five years reading and writing social science academic work, and if there’s one thing I learned it’s to always be skeptical of any finding that confirms the author’s preconceptions. (See also: Stephen Jay Gould’s The Mismeasure of Man — well worth your time if you deal with data.) Data is complicated, any real study is going to generate a ton of it, and seemingly trivial decisions about how to manage it can have dramatic effects on the perceived results.

I spent a lot of time researching education, which shares with software engineering the idea that individual performance is much harder to measure, or even to define, than you might assume at first glance. Empirical education studies tend to fall into one of two groups:

  1. A study under very controlled lab conditions, where the researcher is claiming that a clear data result is applicable to the larger world.
  2. A study in the real world, with messier data, where the researcher is claiming that there is an effect and that it as because of some specific set of clauses.

Both studies are problematic — the first kind often have small or non-representative subjects, the second kind is often a long-term study of one group with real questions as to whether the result is at all reproducible. On top of which you have the Hawthorne Effect (any group that knows it is being observed tends to increase performance no matter what the intervention is) and the effect whose name I can never remember where the more attention is paid to a specific metric the less reliable that metric becomes as a proxy for overall performance.

Or, looking at this another way… I got in a conversation at SCNA this year about why SCNA talks so rarely have empirical results about the value of the software techniques discussed. My kind of glib answer was that we’re all a little afraid that empirical results wouldn’t support the value we perceive in what me might call the “SCNA way”. By which I partially mean that we’re afraid that a badly-designed study might suggest that, say, Test-Driven Development had little or no value, and we’d all have to expend energy dealing with it. (But of course, I’d say that any such study is badly-designed, because of confirmation bias.)

But I also mean, I think, that I’m not interested in that kind of empirical testing and as interesting as I find the pursuit, I have little confidence that it will have much relevance to my day-to-day work. Agile methods, TDD, what we call “good” coding practices make my job easier and more sane. I have my own experience to draw on for that — which I realize is not science, but it’s working for me. Asking them to be proven to be the most efficient way to design software seems like impossible icing on the cake. For me, it’s enough that the methods I favor seem to result in saner, more pleasant work environments. It’s weird to simultaneously be interested in empirical results in my field, and yet at the same time feel they are utterly separated from what I do, but that’s where I am.

Thursday
Nov152012

What's Up?

Oh yeah, things happening…

Got A New Job, Got A New Office

First off, I’ve started a new job as a Senior Developer and Agile Coach at Table XI, putting me back in the realm of small, Chicago-based consulting companies. Very exited, Table XI seems like a great place so far. Hoping to do new things, challenge myself, and learn.

Book Update

Most importantly, you can still buy it.

It’s going slowly but steadily, thanks for asking. I’m determined to start releasing weekly even if it means that I cut off the book in mid-sentence. Hoping that will start next Monday. The remaining Backbone chapter is something like 1/3 to 2/5 done.

Next up, Ember. I was already placed on a blog post of Ember.js resources, which is quite flatteringly optimistic given that the Ember stuff doesn’t technically exist yet. A couple of members of the Ember team have reached out and offered some technical assistance, all of which is very nice, and motivating, and terrifying.

Anyway, the Ember stuff is going to be interesting because, first off, Ember is still a rapidly moving target. I had hoped they’d have locked down a 1.0 API by now, but given the comments made on the recent JS Jabber podcast, it looks like we still have some movement coming. Also, it seems as though the nature of Ember implies a more complex example to really show it off. Still hopeful that I’ll be able to start pushing out Ember content in December, though.

Cons

The video from WindyCityRails is now up, as well as the slides from the evolutionary decendent of the talk from RailsConf — the RailsConf video should be up in a couple of days. There’s a blog post in progress exploring the common theme of those talks, as well as a couple of speakers at SCNA who came at similar themes.

Friday
Oct262012

Velocity, Agile Estimation, And Trust

Charles Max Wood posted this on Twitter:

After trying like six times to fit my response in a tweet, I gave up and remembered that I had this web site where once upon a time I wrote things that were more than 140 characters.

Disclaimer: I have no idea how Charles’ team is working and what might have been said in planning meetings or anything else.

That said, here’s what I think.

The goal of agile project management is accepting the inevitability of change through continual feedback, continual improvement, and a realistic sense of progress. In an agile project, things that are hard when done in bulk — testing, integration, estimating — are done continually, in smaller pieces, to reduce complexity and risk.

Yes, in a functional agile project, velocity is set by team pace. Ideally, you had a meeting at the beginning of the sprint where you estimated velocity based on past performance, and determined the stories you hope to get done based on point estimates of the stories and that velocity. (Point estimates, remember, are measures of complexity, not time…)

It’s possible to be behind in a sprint, in some sense, if it doesn’t look like you are going to complete the agreed upon stories. For example, a story may turn out to be way more complex then estimated. Or a bug may have turned up. (Although agile point estimates are robust against bugs as long as bug fixes remain a roughly consistent percentage of your time).

Remedies for this problem might include changing the point value of a story when new information is determined, splitting the story, lowering velocity estimates going forward, moving a story to the next sprint. Retrospective meetings are a great place for trying to figure out why a story was mis-estimated.

However, trying to assess the state of the project in the middle of a sprint can be a little misleading. There can be a kind of optical illusion if a lot of stories start at the same time, where progress is being made but not booked because the stories aren’t finished. (Sometimes this means you need more granular stories.) Often, it’s helpful to organize the daily standup by outstanding story to give visibility to how stories are moving.

Charles is right that one of the points of Agile project management is not to work by wishful thinking of when you hope things will get done. If a story is more complex than we thought, then it just is, and you need to adjust to that by dealing with the software triangle — change scope, change time, change budget.

Granted #1: This requires a fair amount of trust between the team and the management that when the devs say something is taking longer than expected, that’s assumed to come from a place of expertise, not ignorance. You build trust by being right, and admitting it when you are not right.

Granted #2: Velocity and story points are robust against underestimating, as long as you are consistent. What will happen is that your velocity will settle at a point that factors in the overestimate. However, if specific stories or types of stories are continually taking more time than expected, it’s worth trying to figure out why. If for one reason or another, you aren’t the expected velocity isn’t being allowed to settle to a new state, that’s where the wishful thinking comes in. (Although if your actual velocity is continually dropping, that indicates a problem, too.)

Granted #3: Sometimes there are really are business needs for particular deadlines. That doesn’t change the laws of software physics, but it does determine what a reasonable response is from the team.

Okay, that’s more than 600 words, and I don’t know if I’ve answered the question.

  • In an agile project, velocity should be related to past performance, not hoped-for results.
  • If velocity is being determined top down — for example just trying to determine the velocity by guessing total story points in the project divided by sprints before deadline — that’s not really in the spirit of the thing.
  • It’s possible to miss expectations for a sprint, but the appropriate response to that is usually not “type faster”.
  • This all critically depends on trust between the project managers and engineers.

Wow, it’s been a while since I blasted out a blog post that quickly. Felt good. Hope this helps.

Tuesday
Sep252012

Functions that return functions are the luckiest functions in the world

Here’s some JavaScript:

var foo = function(a, b) { return a * b };

var bar = function(func) {
  return function() {
    return func.apply(this, arguments)
  };
};

baz = bar(foo);
console.log(baz(2, 3));

What we have here is a function, bar, that both takes a function as an argument and returns a function as it’s result. It’s transforming the function passed to it into a different function.

Okay, that’s cool in kind of an abstract way, but so what? Let’s play around with function that take and return functions to show off some potentially useful tricks.

I’ll note right here up top, that there are more complex, fully featured, and robust versions of all the things I’m showing here. I’m just playing with functions.

We can add diagnostics.

var foo = function(a, b) { return a * b };

var trace = function(func) {
  return function() {
    console.log("calling " + func);
    console.log(new Error("normal trace").stack);
    return func.apply(this, arguments)
  };
};

foo = trace(foo);
console.log(foo(2, 3));

All the trace function is doing here is logging a couple of things before actually calling the function — namely, it’s printing the function body (JavaScript doesn’t have a consistent way to get the function’s name), and a stack trace. The console output is:

calling function (a, b) { return a * b }
Error: normal trace
  at file:///Users/nrappin/Dropbox/coderx/blog/functions.html:8:17
  at file:///Users/nrappin/Dropbox/coderx/blog/functions.html:15:13

Okay, that’s kind of neat. You can also tweak it to do something like a Jasmine spy object, but it takes some object manipulation. Here we really start to take advantage of the fact that JavaScript functions are basically just JavaScript objects.

var foo = function(a, b) { return a * b };

var spy = function(func) {
  var returnFunc = function() {
    returnFunc.timesCalled += 1;
    returnFunc.args.push(arguments);
    return func.apply(this, arguments);
  }
  returnFunc.timesCalled = 0;
  returnFunc.args = [];
  return returnFunc;
};

foo = spy(foo);

console.log(foo(2, 3));
console.log(foo.timesCalled);
console.log(foo.args)

Same basic idea here, except we’re doing some dancing with scope to make sure we have access to all the variables we want to, when we want them. Specifically, rather than just returning an anonymous function, we’re giving it the name returnFunc. Inside the funtion body of returnFunc, we’re accessing properties of the returnFunc functional object itself. Specifically we’re incrementing a counter and adding to a list of called arguments.

After we define the function, we initialize the timesCalled and args properties, and return our functional object. This works because we can access properties of the functional object inside the function itself, which seems weird, but makes more sense if you think of it just as an ordinary object accessing other properties of the same object.

Here’s a related example — in this case, our function object has its own methods.

var benchmark = function(func) {
  var returnFunc = function() {
    returnFunc.startDate = Date.now();
    result = func.apply(this, arguments);
    returnFunc.endDate = Date.now();
    return result
  }
  returnFunc.elapsedTime = function() {
    return returnFunc.endDate - returnFunc.startDate;
  };
  return returnFunc;
}

foo = benchmark(foo);
console.log(foo(2, 3));
console.log(foo.elapsedTime());

In this one, we’re running the inner function surrounded by time stamps, and we’re adding a function to the function object, which again, seems weird. At the end, we can query a benchmarked function as to how long it takes. (Yes, I realize that the dev tools allow you to do this.)

Here’s one that might be more practical. Memoizing is the act of caching the results of function calls so that you don’t need to re-evaluate an expensive computation.

var memoize = function(func) {
  var returnFunc = function() {
    argsToCheck = (JSON.stringify(arguments));
    if (returnFunc.cache[argsToCheck]) {
      return returnFunc.cache[argsToCheck];
    } else {
      result = func.apply(this, arguments);
      returnFunc.cache[argsToCheck] = result;
       return result;
    }
  }
  returnFunc.cache = {};
  return returnFunc;
}

foo = memoize(foo);
console.log(foo(2, 3));

In this case, we hold on to the functional argument as before, but when we call the memoized function, we first compare the argument list against a cache. We’re using the JSON string version of the argument list, so we do have that conversion overhead. If this particular argument list has already been called, then we return the result of the previous call without recalculating. If not, we calculate the function, and place the result in the cache. So, our first call to foo in the example places (2, 3) in the cache, but the second call would retrieve it, to avoid that very expensive multiplication.

This is the beginning of manipulating functions in JavaScript — functional programming in general is a big topic, but worth looking at. Adapting a functional style where side effects and mutable state are downplayed in favor of manipulating functions, can really pay off in less complexity in your code.

Like this? You might like my book Master Time and Space With JavaScript, free sample available, and $15 for the whole thing.

Friday
Sep142012

Master Space And Time Status Update

Here’s a quick status update on Master Space and Time With JavaScript, book 3…

Short version: expect an early beta of about 1/3 of the book to be out in about a week.

Longer version:

Work is proceeding steadily, the current draft of just hit 30 pages toward a target of around 90, though about 10 of that is the same intro and outro that the other books have, so it’s more like 20 pages toward a target of 80.

The structure of Book 3 is one chapter on replacing the existing app homepage with a very simple Backbone clone, then another chapter building a different page using Backbone that has more events and more complexity, then a third chapter talking about communication between Backbone and the server, and anything else that comes up. The exact division between the second and third chapters may change.

I will release the first beta when the first chapter is draft complete — this will be rougher than the other books because it hasn’t gone through rounds of review and editing, but I like how it’s going. I also need to clear through reported errata on the first two books, and probably clean up the landing page in expectation of doing another marketing push.

Thanks for being patient and for your great feedback.

Oh, and you can still buy it!

Tuesday
Aug212012

Depending on jQuery and Perspective

The reported errata for Master Time and Space With JavaScript (buy it here) has been pretty light so far. A bunch of typos, some setup weirdness.

And one interesting issue worth exploring. What is a dependency, and maybe more to the point, where is a dependency?

This issue was raised by a reviewer whose name I’m not going to mention — that’s not a reflection on the reviewer, but rather a reflection on the fact that I’m going to put words in his mouth to expand on his brief comment on the issue, so my interpretation of his position may not actually be his position.

Anyway the reviewer had a comment about when to convert to and from jQuery objects. He raised it in the context of the autocomplete example from the please-go-pay-for-it Book 2, but it also applies to the toggler example in just-for-free Book 1, and really, in any case where you have a JavaScript object which uses a data type that is a jQuery object.

Here’s the deal. I have an object — in this case it’s my autocomplete selector, but I’ll pretend it’s a generic widget because the exact object doesn’t matter — which is being initialized via a function call like so.

    widget = new Widget({parentSelector: "#autodiv"});

The key point here being the parentSelector: "#autodiv" option. In the eventual constructor, that #autodiv is immediately converted to a jQuery object (The example in the book is more elaborate, I’m simplifying to focus on the issue at hand…)

    var Widget = function(options) {
        this.$domParent = $(options.parentSelector);
    };

The reviewer’s point was that he’d rather convert the selector to jQuery in the call to Widget and pass the argument to Widget already converted to a jQuery object, rather than have the Widget constructor do the conversion:

    widget = new Widget({domParent: $("#autodiv")});

    var Widget = function(options) {
        this.$domParent = options.domParent;
    };

I’m not convinced yet that one way is better than the other — I haven’t changed the book code, but I certainly respect this reviewer’s opinion on how to structure JavaScript. But I do think that the two options make different assumptions about the structure of the application that are worth teasing out.

In my structure, jQuery is treated as an implementation detail of the Widget. Since jQuery is an implementation detail, outside code wouldn’t be expected to know about it, so outside code communicates to the Widget via a selector. Now, I grant that there’s a little bit of fiction there, since the selector syntax is (somewhat) jQuery specific. If I really wanted to isolate jQuery, then the calling argument should just be the DOM ID, and the Widget would also be responsible for converting the DOM ID to a jQuery object, with the intermediate step of building a jQuery selector string:

    widget = new Widget({parentId: "autodiv"});

    var Widget = function(options) {
        this.$domParent = $("#" + options.parentId);
    };

The advantage of treating jQuery as an implementation detail of my widget is that the rest of my code does not need to know or care about it, and I can presumably even test other parts of my code without jQuery even being around. Also, if I choose to swap jQuery out for some reason, the rest of my code doesn’t need to know, only my widget needs to be changed. I would consider the conceptual point about jQuery being an implementation detail of the widget to be important even if I find it exceptionally unlikely that I would swap out jQuery. (For one thing, it also protects me against jQuery itself changing).

In my reviewer’s structure jQuery is a dependency of the application as a whole. Looked at from that perspective, it makes sense to convert everything to jQuery objects as early as possible, to maintain consistency of representation across the entire application. The code as a whole may be easier to read, since we aren’t continually having to worry about whether an object is a jQuery object or just some kind of DOM identification. If multiple widgets are all using the same jQuery object, then we might prevent some duplicate conversion to jQuery objects. This probably simplifies the internal code at the cost of making us more dependent on jQuery itself. As a practical matter, that tradeoff might be worth it — once we’ve decided to use jQuery, changing it is probably unlikely.

Essentially, it’s a question of where you draw your boundaries. I’m not sure there’s a long-term practical difference between these two structures, in that I don’t think one of them necessarily leads to better or more flexible code over time, especially given even rudimentary attention to practical details. But I do think you should be clear about which structure you are using — mixing the two by treating jQuery as a specific dependency of some parts of the code but a general dependency of other parts would probably lead to confusion later on.

Update:

A reddit commenter asked why I wasn’t passing a DOM element into the Widget, as in:

new Widget({parentSelector: document.getElementBy("autodiv")});

My response: I have nothing really against passing in a DOM element beyond it being a little verbose and an old-timey superstition against using getElementBy. Using a DOM element still keeps the dependency in the widget.

Like this? Go ahead and read Master Time and Space With JavaScript

Thursday
Aug092012

How's it Going? MSTWJS Edition

And now for a more inside-baseball post about how the self-publishing aspects of Master Space and Time With JavaScript are going. Did you know you can buy it?

Short answer: Pretty well, though I could always have done better. Still unclear how this will work over the long haul.

At this point, the book has been on sale for 10 days, plus the pre-sale to people who were on the mailing list. It’s clear that the initial burst of traffic from incoming links is slowing down, and I’m now entering the longer struggle to get people interested — not completely sure how to do that.

Anyway, a few disconnected points about the process so far

One of the big things I miss about a larger publisher is the marketing reach. That said, there’s something really nice about how people feel a little bit of ownership concerning self-published projects that they like. I’ve gotten a couple of very nice copy-edit runs, for example.

I’ve generally been lucky in reviews (notable exception: the two-star review of RTP on Amazon that I check out whenever my Impostor’s Syndrome feels insufficiently pronounced…), and so far, the people who have commented on the books where I can see them have been positive.

I can be a little transparent with numbers. As I type this, traffic is still about double the level that I had generated in the past on days that I posted to the blog, and much, much higher than the ambient level of traffic when I hadn’t been posting.

Over the course of the ten days, about one in six people that have hit the landing page at http://www.noelrappin.com/mstwjs and aliases convert to either the free version or one of the paid versions. Low day was 12%, high day was just under 23%. There doesn’t seem to be a consistent trend between traffic and conversion rate.

So that’s about one in six doing anything, of those that do choose, about one in six or so actually bite on one of the paid versions. So that’s a paid conversion rate in the 2 - 3% range. That rate seems to be slightly negatively correlated with traffic, which is actually in line with what I would expect.

As for the pricing strategy, which I thought was so clever… So far, there have only been a very few people buying book 2 at the $7 level — most purchases have been of the whole book at $15. I’m not planning on changing the pricing (beyond my already-stated plan to raise the $15 when Book 3 gets near-final). I want to see how this looks when there are more individual books for sale. But there’s a good chance this means that I outsmarted myself, and probably could have priced a little higher.

So far, as best as I can tell, under 3% of people who originally downloaded the free version came back to upgrade. That seems very low, but it might be higher — the way I’m counting this, if somebody gave me a bogus email for their freebie, I wouldn’t track it as an upgrade. Also, I assume this number goes up a touch as people read the book and as upgrade reminders go out. Still, as it stands, it’s not a great data point for the “give away free stuff to increase sales” school of internet marketing.

Over the course of the ten days, the number one referrer, by far, was Peter Cooper’s JavaScript Weekly email newsletter. It’s the biggest by about a factor of three over the next highest measurable referrer. Next up was Twitter, and I think the highest link out of those came from JavaScript daily, and I think the second one was from my feed. It’s really hard to track those for sure, though. Third was Google Reader, though I think that was mostly blog posts and not links to the landing page. Fourth was Reddit — my post there didn’t get much traffic, and fifth was the Ruby5 podcast and show notes. Rounding out the referrals so far is Mike Gunderloy’s Fresh Cup links, and then we also get some noise with internal referrals and things like Pocket, and the Ruby Rogues link, which just came out.

Another point of comparison is that MSTWJS sales are about 150% + of RTP sales over the first ten days. That’s less impressive than it sounds, I really struggled to get traction with RTP after the initial burst. (That’s the paid number for MSTWJS, and it includes the pre-sale to the list). RTP had a free section as well, and I don’t have any stats on how often that was downloaded, but since it was just a link and not a shopping cart, I think it was pretty high.

That’s where we are — I hope that those of you that bought or downloaded the book are enjoying it, and if you are hoping to do your own self-publishing project, I hope this information is helpful.

Oh, and the book is still on sale.

Monday
Aug062012

The Origin of Master Space and Time With JavaScript

I have a new book, Master Space and Time With JavaSript. You can buy it.

Here’s the secret origin.

This all started over a year ago. Rails Test Prescriptions had been complete for a few months, and I was getting a little antsy to take on a new project.

But what? I wanted it to be a project where I would learn something, and I wanted it to be something where I had a particular perspective to offer.

A couple of recent experiences pushed me toward view-level coding in general, and JavaScript in particular. About then, for the first time in a while, I worked on a project that had a reasonably serious JavaScript front-end on a team small enough that I was contributing some of the JavaScript.

It quickly became clear to me that JavaScript coding had changed dramatically, not just from my first pass at it (warning clients away from it circa 2000), or my second pass (avoiding it with RJS circa 2008). The tools were better, the idioms had changed, and the expectation was that future web applications would need to handle this stuff very well.

I was also working with a new Obtiva apprentice, who wanted to build a very JavaScript-heavy site, but didn’t really know much JavaScript. In searching for books to point him to, it seemed to me like there was an underserved part of the market, not for total beginners, not a description of a particular library, not a pronouncement from On High about The Right Way To Do Things, but a practical guide to writing and testing modern JavaScript to do cool stuff.

Which is what I set about to write. My original proposal for this book may well be one of the best pages of text I’ve ever written.

And yadda, yadda, yadda, here we are.

I knew I wanted the book to build up on a single web application example — I’ve always liked that style, even though it can be a pain in the neck to structure. I also knew that I wanted to have a lot of testing in the book. Not only is writing about testing in JavaScript something that seems pretty needed, it also felt like a perspective where I actually might have useful things to say. Plus, I had used the largely-test-first style of writing before (in the Wrox book), which I’m sure was appreciated by all three of the people who bought it. I thought I could do it again.

The idea of taking an application with no JavaScript and adding JavaScript features seemed like a good hook. I’d used the Time Travel Adventures travel agency before (for my testing legacy code workshop), and the idea of a time-traveling client who was confused about what modern web sites needed seemed like a suitably silly hook. (In the first draft, the client was named Emmett Brown, but I was guided away from using that name directly, hence the client becomes the mysterious Doctor What.)

And I was off…

Here’s what you get in the books.

Book 1 is largely concerned with a particular simple-seeming request: add a show/hide toggle link to each trip on the home page. Although this is simple, it actually winds up touching a fair amount of jQuery — using selectors to find elements, binding events, and manipulating element pieces.

We build up this toggle thing in a few stages. First is a quick pass writing a simple version of the toggle functionality test-first. The goal here is a sense of what a test-driven process looks like in JavaScript, plus the basics of how Jasmine and jQuery can be brought to bear to write the feature. It’s basically the book equivalent of “fast to green”. We solve the problem with the understanding that we’ll clean up the details later.

Then we go back with two more chapters that go more in-depth on first Jasmine, then jQuery. In these chapters, we don’t add new features as much as describe the library features that supported what we did and explore related functionality.

The final chapter of Part 1 covers the JavaScript object model and why it is confusing if you come to JavaScript from a more traditional Object-Oriented language. By the end of it, we’ve used the module pattern to build the most over-designed show/hide toggle ever. I’ve had some experience doing workshops based on the material in this chapter, and it seems like even people who have been doing JavaScript for a while get something new out of it.

And that’s Book 1. It’s available for the low, low price of zero dollars and zero cents. Worth every penny.

Book 2 is mostly about apply and extend. The first chapter is about building up an widget that combines an autocomplete text input with a list of currently chosen items. Test-first, of course. The we throw some Ajax in the mix, extending our already gold-plated toggler with the ability to get data from a server. This also gives us an excuse to talk about Jasmine spies. Finally, we build up a rating widget, with the clickable stars and a histogram and stuff, which lets us talk about JSON and Mustache. There’s also a small, slightly out of date bit on the Chrome developer tools. I’ll catch up on that at some point.

Book 2 is a mere $7.

Or you can get Books 1 & 2, Plus books 3 & 4 when they come out for $15. That $15 is a temporary price, and will go up when Book 3 gets closer to completion.

Book 3 is going to be about Backbone. I know the structure of most of it — first, we’ll recreate the front page using a Backbone structure. Next, we’ll build a buy page that allows you to make several calculations about differing trip purchase options on the client. Not sure about the last part, it will definitely include communicating back to the server, probably something else.

Book 3 is going to start appearing on the site in a couple of weeks, and will probably be draft complete by the end of September.

Book 4 will cover Ember.js. I’m not sure yet what we’ll build, though I want it to be a new part of the site, and not a recreation of the same things we do in Backbone. I’m hoping that Book 4 will be out by the end of 2012.

Oh — the title. I had a list of two word titles that were like POWERWORD JavaScript, but all of them were either taken, sounded ridiculously stentorian to me, or both. The original proposal titles were “Getting Things Done in JavaScript”, which nobody liked, or “JavaScript for People who Hate JavaScript”, which nobody liked (see the pattern).

I had Master Space and Time with JavaScript on my list as kind of a joke — a reference to the Time Travel conceit in the book. Plus I liked that it sounded like a pulp adventure novel, and would lend itself to a cover easily. I know it’s not the, like, SEO favorite title, but I’m hoping that people won’t forget it once they hear it.

The cover, by the way, is my own design, and I like it considerably more than many of the other things I’ve designed. (I’m also pretty happy with the PDF layout…) I think the cover particularly works well at thumbnail size, so you can see the difference between the individual books easily.

That’s the story. Hope you like it. Buy it!, or tell all your friends (Twitter hashtag #mstwjs).

Thanks.

Monday
Jul162012

Rails, Objects, Tests, and Other Useful Things

For the first time in quite a while, I’ve been able to spend time working on a brand-new Rails application that’s actually a business thing and not a side project. It’s small. Okay, it’s really small. But at least for the moment it’s mine, mine, mine. (What was that about collective code ownership? I can’t hear you…)

This seemed like a good time to reflect on the various Object-Oriented Rails discussions, including Avdi’s book, DCI in Rails, fast Rails tests, Objectify, DHH’s skepticism about the whole enterprise, and even my little contribution to the debate. And while we’re at it, we can throw in things like Steve Klabnik’s post on test design and Factory Girl

I’m not sure I have any wildly bold conclusion to make here, but a few things struck me as I went through my newest coding enterprise with all this stuff rattling around in my head.

A little background — I’ve actually done quite a bit of more formal Object-Oriented stuff, though it’s more academic than corporate enterprise. My grad research involved teaching object-oriented design, so I was pretty heavily immersed in the OO documents circa the mid-to-late 90s. So, it’s not like I woke up last May and suddenly realized that objects existed. That said, I’m as guilty as any Rails programmer at taking advantage of the framework’s ability to write big balls of mud.

Much of this discussion is effectively about how to manage complexity in an application. The thing about complexity, while you can always create complexity in your system, you can’t always remove it. At some point, your code has to do what it has to do, and that’s a minimum on how complex your system is. You can move the complexity around, and you can arguably make it easier to deal with. But… to some extent “easier to deal with” is subjective, and all these techniques have trade-offs. Smaller classes means more classes, adding structure to make dependencies flexible often increases immediate cost. Adding abstraction simplifies individual parts of the system at the cost of making it harder to reason about the system as a whole. There are some sweet spots, I think, but a lot of this is a question of picking the Kool-Aid flavor you like best.

Personally, I like to start with simple and evolve to complex. That means small methods, small classes, and limited interaction between classes. In other words, I’m willing to accept a little bit of structural overhead in order to keep each individual piece of the code simple. Then the idea is to refactor aggressively, making techniques like DCI more something I use as a tool when I see complexity then a place I start from. Premature abstraction is in the same realm as premature optimization. (In particular, I find a lot of forms of Dependency Injection really don’t fit in my head, it takes a lot for me to feel like that flavor of flexible dependencies are the solution to my problem.)

I can never remember where I saw this, but it was an early XP maxim that you should try to keep the simplicity 90% of your system that was simple so that you had the maximum resources to bear on the 10% that is really hard.

To make this style work, you need good tests and you need fast tests — TDD is a critical part of building code this way. You need to be confident that you can refactor, and you need to be able to refactor in small steps and rerun tests. That’s why, while I think I get what Gregory Moeck is saying here, I can’t agree with his conclusion. I think “more testable” is just as valid an engineering goal as “fast” or “uses minimal memory”. I think if your abstraction doesn’t allow you to test, then you have the wrong abstraction. (Though I still think the example he uses is over built…).

Fast tests are most valuable as a means to an end, with the end being understandable and easily changeable code. Fast tests help you get to that end because you can run them more often, ideally you can run them fast enough so that you don’t break focus going back and forth between tests and code, the transition is supposed to be seamless. Also, an inability to write fast tests easily often means that there’s a flaw in your design. Specifically, it means that there’s too much interaction between multiple parts of your program, such that it’s impossible to test a single part in isolation.

One of the reasons that TDD works is that the tests become kind of a universal client of your code, forcing your code to have a lot of surface area, so to speak, and not a lot of hidden depth or interactions. Again, this is valuable because code without hidden depth is easier to understand and easier to change. If writing tests becomes hard or slow, the tests are trying to tell you that your code is building up interior space where logic is hiding — you need to break the code apart to expose the logic to a unit test.

The metric that matters here is how easily you can change you code. A quick guide to this is what kinds of bugs you get. A well-built system won’t necessarily have fewer bugs, but will have shallower bugs that take less time to fix.

Isolation helps, the Single Responsibility Principle helps. Both of these are good rules of thumb in keeping the simple parts of your code simple. But it also helps to understand that “single responsibility” is also a matter of perspective. (I like the guideline in GOOS that you should be able to describe what a class does without using “and” or “or”.

Another good rule of thumb is that objects that are always used together should be split out into their own abstraction. Or, from the other direction, data that changes on different time scales should be in different abstractions.

In Rails, remember that “models” is not the same as “ActiveRecord models”. Business logic that does not depend on persistence is best kept in classes that aren’t also managing persistence. Fast tests are one side effect here, but keeping classes focused has other benefits in terms of making the code easier to understand and easier to change.

Actual minor Rails example — pulling logic related to start and end dates into a DateRange class. (Actually, in building this, I started with the code in the actual model, then refactored to a HasDateRange service module that was mixed in to the ActiveRecord model, then refactored to a DateRange class when it became clear that a single model might need multiple date ranges. The DateRange class can be reused, and that’s great, but the reuse is a side-effect of the isolation. The main effect is that it’s easier to understand where the date range logic is.

I’ve been finding myself doing similar things with Rails associations, pulling methods related to the list of associated objects into a HasThings style module, then refactoring to a ThingCollection class.

You need to be vigilant to abstractions showing up in your code. Passing arguments, especially if you are passing the same argument sets to multiple methods, often means there’s a class waiting to be born. Using a lot of If logic or case logic often means there’s a set of objects that have polymorphic behavior — especially if you are using the same logical test multiple times. Passing around nil often means you are doing something sub-optimally.

Another semi-practical Rails example: I have no problem with an ActiveRecord model having class methods that create new objects of that model as long as the methods are simple. As soon as the methods get complex, I’ve been pulling them into a factory class, where they become instance methods. (I always have the factory be a class that is instantiated rather than having it be a set of class methods or a singleton — I find the code breaks much more cleanly as regular instance methods.) At that point, you can usually break the complicated factory method into a bunch of smaller methods with semantically meaningful names. These classes wind up being very similar to a DCI context class.

Which reminds me — if you are wondering whether the Extract Method refactoring is needed in a particular case, the answer is yes. Move the code to a method with a semantically meaningful name. Somebody will be thankful for it, probably you in a month.

Some of this is genuinely subjective — I never in a million years would have generated this solution — I’d be more likely to have a Null Object for Post if this started to bother me, because event systems don’t seem like simplifications to me.

I do worry how this kind of aggressive refactoring style, or any kind of structured style, plays out in a large team or even just a team with varying degrees of skill, or even just a team where people have different styles. It’s hard to aggressively refactor when three-dozen coders are dependent on something (though, granted, if you’ve isolated well you have a better shot). And it’s hard to overstate the damage that one team member who isn’t down with the program can do to your exquisite object model. I don’t have an answer to this, and I think it’s a really complicated problem.

You don’t know the future. Speculation about reuse gains and maintenance costs are just speculation. Reuse and maintenance are the side effect of good coding practices, but trying to build them in explicitly by starting with complexity is has the same problems as any up-front design, namely that you are making the most important decisions about your system at the point when you know the least about the system. The TDD process can help you here.

Friday
Jun082012

Upcoming Me

Updates, schedules, things, and stuff.

Scottish Ruby

The Scottish Ruby conference is having a charity workshop June 28, and I’m presenting my “Advanced Rails Design” workshop. This is the extended dance mix version of the workshop I did at Mountain West Ruby earlier this year. I thought it went really well (so did the attendees, I’m sure), and I’m very excited about this one. Details at http://scottishrubyconference.com/charity/ — you don’t need to be attending Scottish Ruby, but you do need to register in advance.

There are three workshops that day — a UX workshop that has sold out. A Dave Thomas advance Ruby workshop that hasn’t, and mine. Let’s just say there are more tickets available right now for mine than I’d like, and I hope that if you are in the neighborhood, you’ll stop by. It’ll be worth it.

Windy City Rails

Much closer to home, I’m speaking at Windy City Rails this year. According to their schedule, my talk will be “Let’s Make Testing Fun Again”. This conference is always great, the venue this year looks outstanding, and the speaker list is — myself excluded — top-notch. Hope to see you there.

Ignite Rails

My IgniteRailsConf talk: Manage Your Development Environment / Never Burn Another Burger by Noel Rappin is now available on line at http://www.youtube.com/watch?v=VLCLBdFsSOE. I don’t think my other RailsConf thing is up yet, but I’m sure I’ll let you know.

Master Space and Time With JavaScript

It’s near schedule. I think that converting all the text for the first two parts of the book will be done next week. Leaving me with a) a serious edit b) cover and incidental design, c) cleanup for epub and mobi, and d) product sale logistics. Still hoping that’ll go out in early July. If you’ve signed up at http://www.noelrappin.com/mstjs-form, you’ll probably get an early look/chance to help me work the kinks out.

And Hey,

You can still buy Rails Test Prescriptions. Much of it is still up-to-date…

Wednesday
Jun062012

Automator + Bash = Yay

At it’s best, working in Mac OS X combines the power of the Unix shell with the convenience of an actual interface.

Here’s a best case scenario:

As I may have mentioned here a few times, I’m writing a book. As part of my current workflow, I need to convert my text from it’s old format to my new format, which is Markdown. The old format is a custom XML-based language the details of which don’t matter beyond the fact that it’s XML-based.

Moving the text over has two issues:

  • The obvious one is that there are XML tags in the body of the text for things like code literals and text italics that I want to replace with either the Markdown backtick or the Markdown underscore.
  • The less obvious one is that when writing XML text, I treat it like code, meaning I’m ridiculously insane about text layout. In my XML source, I really do still insist on an 80-character line, with hard returns and indentation. I’ve decided that I don’t need to do this when I write in Markdown, so when I move the text over, I need to get rid of all the hard returns and spacing.

What I’d like is a workflow where I can copy a paragraph of text to the clipboard, do magic, and then paste cleaned-up text in the new book file. Going a paragraph at a time is not a problem — it’s actually preferable, since I’m editing as I go, so moving a whole file at a time is not really what I want.

I toyed with the idea of doing this in Ruby, but that seemed like a pain, so I wrote a short shell script.

Understand: I never do this, and I pretty much know beans about the shell programs involved. But by googling things like “shell remove newlines” and with some helpful man pages, I cobbled together the following, with details of the XML fuzzed a bit.

    pbpaste | tr -d '\n' | tr -s ' ' | 
    sed -E 's/<\/*(literal|lit)>/\`/g' | 
    sed -E 's/<\/*(italic|bold)>/\*/g' | pbcopy

All of you who actually know shell scripting are invited to have a hearty laugh. While you are off chuckling, I’ll explain what this line noise does…

  • pbpaste takes the text from the clipboard, which is piped to…
  • tr -d '\n', which removes all the newlines, and pipes to…
  • tr -s ' ', which removes duplicate spaces, and pipes to…
  • sed -E 's/<\/*(literal|lit)>/\/g’`, which takes all the XML tags for things that I want replaced with Markdown literal syntax and puts in a back tick, then pipes to…
  • sed -E 's/<\/*(italic|bold)>/\*/g', takes the XML tags I want to replace with Markdown emphasis syntax, then pipes to…
  • pbcopy, which copies the final text back into the clipboard

Works great. A bit of a mouthful to type, if I can mix metaphors. (The actual one is even more complicated, because I strip out some XML entities as well). It’s a bit much to type at the terminal in my workflow. Creating an alias is a possibility, but still, requires a terminal.

There’s another option in Mac OS X, though: Automator.

You may not have played with Automator, because you do not fully appreciate Automator. Automator is an OS X application that lets you chain together predefined actions (very similar to the actions exposed via apple script), using a GUI interface, and save the result as an application or an OS X service, among other options.

This isn’t an Automator tutorial, because what I want to do here is really simple. One of the available actions in Automator is “Run Shell Script”.

Hmm…

I created a new Automator document as a Service, added the Run Shell Script action to it, pasted my big shell script into the body of the action. The action doesn’t need input. Even better, I can have it work not on the clipboard, but on the selected text in the open application, which saves me a step in my workflow. The shell script is already putting the output on the clipboard, so I don’t need to deal with that in automator either.

Okay, big deal, it runs a shell script. But, since I’ve saved it as a service, I can assign a global key command to it. In the System Preferences for Keyboard, my new service is somewhere in the Keyboard Shortcuts tab (under services). From there, I can assign a keyboard shortcut, which is available in any application that exposes itself to services, which is many applications.

Now I just select a paragraph of old text in my text editor, hit my new key combination, and I can paste the cleaned up text in my new editor window. A little Unix, a little Mac, and a lot of time saved.

Tuesday
May292012

Master Space And Time Release Plan

There is a plan.

It goes like this:

Master Space and Time With JavaScript will be split into four parts.

Part one and part two will be available sometime in July. I’d say July 1st, but I’ll still be in Scotland. It’d be sooner, but there are still logistics to be managed around the actual layout of the book and getting the payment gateway in order. Plus, I need to actually finish the text.

Part one is an introduction to Jasmine, jQuery, and the JavaScript object model. It will be available for free.

Part two will be more advanced jQuery, Jasmine, and JavaScript examples. It will be available for, most likely, $7.

Each part will be on the order of 50 - 75 PDF pages. The exact split between the two parts will depend on the final page count of the chapters.

You will also have the option to pre-purchase the entire book — that’s an immediate download of parts one and two, and an eventual update with parts three and four. That bundle will most likely be $15. I’m not sure yet how long that option will remain available.

Part three, will cover Backbone.js. It will come out something like four to six weeks after parts one and two. Part three will also be $7.

Part four, which will cover Ember.js, will come out some time in the future, same deal.

There are two reasons why the Ember.js part might take a few months. One is that I’m still kind of waiting for Ember.js to settle.

But the main reason is that I plan on doing other things.

More mini-books, in the 30-75 page range, for $5 to $10 each. I’m going to do at least one, maybe two, before the Ember.js part. (Yes, I’ll say what they are. Eventually).

And that’s the real plan: an outlet for me to release these mini-books on a regular basis. To write these ideas that I have that aren’t quite long enough for full books.

I’m excited about this. I have a few topics I’ve wanted to write about for a long time, and this is way for me to get these out to people who I think will find them useful, interesting, and fun.

A couple of other logistical things:

  • In case it’s not clear, all releases will be digital with PDF, ePub, Mobi, maybe HTML. iBooks Author is a possibility, but probably not in the immediate future.
  • I would update previous books to correct errors and typos and that kind of thing.
  • I’d like to have some way to offer something like a subscription. I need to find out what the payment tool I’ll be using makes possible.
  • I’ll probably have some sort of umbrella name for the series, but I don’t know what it will be yet.

I think this is a good plan. I hope you agree. I’d love to hear your thoughts, and if you want to get an email when the first two parts are available, please fill out the interest form

Friday
May182012

Master Space And Time With JavaScript Update: The First Couple of Chapters

It’s not much of an exaggeration to say that I’ve been writing the same two or three chapters for six months. I think — I hope — this is the last time.

This is a more or less weekly update on the manuscript currently known as Master Space & Time in JavaScript. Today, the update is about the first few chapters and how they change over time. These chapters cover a lot of ground, and getting the order and beats right has been a struggle. But I think I’ve got a good handle on it now.

(If all you want is a pure update: I’m making progress. The biggest factor in when the book actually goes on sale is how much I decide to have completed before I sell it. But I think we’re still looking at a few weeks out.)

The problem with the first couple chapters is that there are a lot of topics to cover right at the beginning. The first part of the book introduces jQuery, but I also want to introduce testing as a practice, so Jasmine comes in immediately as well. I want to use the examples in the book to model a good test-first practice (for the two or three of you that are familiar with my Wrox book, you know that I tried something similar there).

However well intentioned my plan to cover testing is, the fact is that for a reader unfamiliar with both, simultaneous exposure to Jasmine and jQuery has a lot of potential to be confusing.

In my first draft, the reader was presented with a suite of three or four Jasmine tests, which were fully explained, then with the jQuery code. The example is super-simple — just making a detail show and hide based on a user click. That’s on purpose, so nobody gets bogged down in the details of the example. Even though it’s simple, it does touch on a lot of basic jQuery features, which were explained in some detail, while also referring back to the Jasmine.

Some early reviewers found the back and forth confusing, so I tried a few different ways to clean it up:

  • I really like starting books with quick demos that show off things that will be fully explained later. So I added one that covered the JavaScript console. More on that in a moment.
  • I added a lot of text explaining exactly where the reader was, why we were talking the thing we were talking about and so on. A lot of this was necessary and stuck around, though at one point I wound up with a two-page chapter introduction that was — accurately — described as “apologetic”.

Eventually, though, I tried to separate the chapters, presenting the jQuery code first and then the Jasmine tests after. On the plus side, separating the two led me to write much better explanations of each library. On the down side, the order just felt really wrong to me, it wasn’t really the way I wanted the book to go.

As I tried to bring the material together for the new version, I came up with an answer that I think keeps all the draft material I like, and scraps the stuff I don’t. I mentioned that I really like quick-march introductions for my books. It finally occurred to me that the small example actually works as a quick introduction. So the plan is to work through that example in a kind of strict test-first way, showing the rhythm of a BDD process without a lot of library detail. Then go back and describe Jasmine in more detail, then jQuery in more detail.

Which makes this the current table of contents:

  • Introduction. Mostly logistics, explanation of what’s there and so on.
  • First Look: The quick walk through Jasmine and jQuery.
  • Jasmine in more detail.
  • jQuery in more detail — at least the basic selector and element manipulation parts.
  • JavaScript functions and objects. How they work. We take the simple code from the first example and apply different JavaScript module and class patterns to it. I like this chapter.
  • Developer tools, the console and the WebKit package
  • Pulling the first few chapters together on a more complex example — a multi-select autocomplete widget.
  • Ajax. How to do it and how to test it.
  • JSON. This is an Amazon-style rating widget, so pulling in everything so far plus Ajax, plus JSON.
  • Backbone. Probably 2-3 chapters here, covering Underscore, and Mustache. This is the point where we get to things that haven’t been written yet, though I do have an outline.
  • Depending on how ambitious I feel at this point, a similar treatment of Ember.js is possible.

The initial release is most likely everything up to the JSON chapter, thought it’s possible I may start putting it out there once I finish reworking the autocomplete widget chapter.

If this sounds interesting to you, let me know by signing up. There’s still time to suggest other tools you want covered. Or leave a note in the comments here.

Monday
May142012

Self-Publishing Workflow Update

Next up on the Master Space and Time With JavaScript status report is the workflow that takes my words and turns them into a PDF. And an HTML file. And an ePub. And don’t forget Kindle.

As you can imagine, this is something of a minefield, although there are a lot more tools available than there were three years ago when I did this the last time — here’s an overview of the process I used then. That article talks about the process that I used on Rails Test Prescriptions for as long as it was self-published.

Things have gotten more complicated. Most obviously, there are more devices and formats to support. The Kindle’s mobi format and ePub are different, and every ePub device has its own quirks. On the plus side, there are a lot more tools and libraries available than there were, though figuring out what they all do is a challenge in itself. Plus, a lot of the existing tools produce documents that are, well, kind of dull-looking, especially noticeable in the PDF versions. (If you are a self-publishing author, I don’t mean you, your stuff looks great. Those other people, though… can you believe them?)

Over the last few years, I’ve gotten addicted to being able to sort of see what the text looks like all laid out fancy and the like. So it was important to me to get at least the semblance of a tool chain in place before I started rewriting in earnest, not least because I needed to figure out what format the text was going to be.

Here’s what I’ve got. It starts with Markdown, because Markdown is pretty simple, I’ve been using it for years, and there are like a jillion editors that support it. Because I’m using Markdown, I’m also experimenting with writing the book in a more writerly editor like Byword, rather than a programmer editor like Sublime. In theory, this will make the book less… I don’t know, less programmerish, or less sublime? Dunno, but right now, I’m enjoying the change of scenery.

Then it starts to get complicated. There are multiple implementations of Markdown or Markdown-like libraries, and they all are subtly different. Right now, I’m using Multi-Markdown, since it has footnotes and cross-references, though there’s some possibility I’d switch at some point in the future. (I love footnotes, and I’d add one here, but I don’t think this blog engine uses a version of Markdown with footnotes.)

Markdown’s weakness is that it’s hard to specify custom styles or HTML classes. I’m working around that by running my Markdown text through a pre-processor where I can put in some custom directives. I can, for example, write:

!!!sidebar
:title A Sidebar
The sidebar content goes here

The pre-processor catches that, and converts the code into some specially styled HTML. (Yes, I’ll be open-sourcing the tool at some point in the future.)

I’ve also added something that I’ve wanted for a long time, namely the ability to insert code from a file at an arbitrary branch in a git repository, meaning that I can show code from multiple successive versions of the same file just by having them in different git branches. Meaning I can distribute the sample code in a git repository and have it not look too awkward.

I also have the ability to post-process Markdown’s HTML, which I think I’m eventually going to need to reconcile the rest of the tools.

Once I have the converted HTML, I need to generate e-book files. For PDF, I’m going back to using PrinceXML. There’s a lot of things I like about this tool. For one, it has a lot of the features that you would normally associate with actual books, like section numbering, footnotes, cross references, and page headers and footers. I think it produces nice-looking files, and its controllable with CSS, a technology that I (mostly) understand, so it’s easy and fun to tinker with.

I didn’t have any existing tools for ePub or Mobi, so I looked at somebody who I thought was creating pretty nice files, specifically Avdi Grimm, who did a pretty great job with the Objects on Rails ebooks. Avdi created his own tool called OrgPress, and for ePub and Mobi, he uses the command line interface to Calibre. Even though OrgPress uses Emacs Org Mode, Make, and Awk — three tools that, to put it mildly, I feel no particular pull to tinker in — I was able to take advantage of Avdi’s hard-fought war with Calibre to get a set of command line flags that basically work, though I’m going to have to tweak them a bit before they are salable. I’m also looking at Rpub, since it’s in Ruby.

One nice side effect of all these tools is that it’s easy for me to have an almost all iPad workflow — I turn on watchr on the laptop, write using Byword on the iPad, and when Dropbox syncs the file, the ebooks are all updated, and I can view them on the iPad via Dropbox. Not bad.

I know this isn’t done yet. For one thing, I suspect that Multimarkdown and PrinceXML are going to disagree on the format for a footnote, and I’m probably going to have to referee. Later note: it’s worse then that… Multimarkdown’s footnote format actually crashes the Mac Adobe and Nook ePub readers, though that seems to be a bug on their end, and it does work in iBooks.

I’m eventually going to need custom styles for each format — the sidebar CSS that looks great in PDF looks awful in iBooks. And I’ll need a cover.

And, you know, content.

Wednesday
May092012

May 9, 2012: The Random Link Post Returns

And now, the return of the semi-occasional link post. I’m going to try to do this at least once a week, but who knows.

If you are writing JavaScript, you should be looking at Justin Searls and his JavaScript testing tools. Justin posted the slides for what looks like a great talk on JavaScript testing. These slides made me happy.

In random media sales, the audio book of World War Z is on sale for a mere six bucks.

A couple of Ruby posts. Steve Klabnik argues that merely splitting code into modules doesn’t reduce complexity. Instead he argues that you need encapsulation. I think splitting code is probably better than nothing, but not a lot better.

Meanwhile, Avdi Grimm describes the Ruby idiom for type conversion which I have to admit, I never thought of as an idiom before.

In a story that I think of as a cautionary tale about pricing and value, the LA Times writes about the history of American Airlines customers who bought unlimited tickets. And then, you know, they used them, unlimitedly.

I always like to see plucky programmers trying to self-publish books about testing. So I’m glad that Aaron Sumner is LeanPubbing a book on testing Rails and RSpec. Good luck, Aaron!

Pretty much everybody who blogs or writes or tries to explain things to people should read this XKCD

Finally, a random music recommendation. I don’t recommend music much, but I do have a weakness for lyric-heavy, earnest, catchy music. Which brings me to the Lisps and their recent musical project Futurity. The musical is a Steampunky kind of thing that concerns a Civil War vet who tries to build a “Steam Brain” with the help of Ada Lovelace. It’s clever and I like it. Album Link. On a similar vein, their song Singluarity from their previous album Are We At The Movies.