The designer sweet spot: Gut driven design that’s data informed

Gut Driven vs Data Driven

Very few organizations exist that are primarily data driven when it comes to design. These organizations allow data to call the shots. They’ll head whichever direction the data points them.

Most organizations these days seem to skew heavily to the other side of the spectrum. They rely almost exclusively on their gut (plus experience) to make design decisions.

Where do you stand?

It’s natural to feel polarized toward one side or the other. Many organizations are either all in on data driven design, or zealots about listening to their gut.

You already know from the title of this post where I stand. I believe pretty strongly that there’s a sweet spot, and I don’t think it lies at either end of the spectrum.

In my experience, the sweet spot comes when a designer primarily trusts their gut to design, but also allocates some time to collect data, and to do some testing. These last two things are key, because they help you validate your assumptions, and they add clarity to your design process.

Trusting your gut is paramount

Great design comes primarily from a combination of experience, and from a designers gut intuition. That said, at what point do you know (not think, but really truly know) whether a design is great?

Truthfully, many designers and organization never take the time to find out. They’re content to ship whatever design they think is best. End of story. But that’s a shame. Launching designs without testing is just ignorant, lazy, or both.

Think about it. You’d never intentionally launch a design that stinks (at least I hope you never would). But chances are that some of the designs you launch — even though you may feel they are great — may actually stink. If you aren’t testing your designs, you’ll never really know which of your designs are the great ones, and which ones could still use some love. Without testing your design or looking at any data, all you really have is a hypothesis that the design you’ve created is great.

Clarity comes via data & testing

Just mentioning the word “data” can brings a sour look to some designers faces. But it shouldn’t. Data in and of itself isn’t bad. Data when leveraged properly can lead to insights and clarity.

It’s worth noting that there’s a huge difference between being data informed, and data driven. Data’s only role should be to inform – never to drive design decisions.

So far, I’ve been ranting about incorporating some “data” and “testing” into your design process, but what does that really mean?

Let’s look at a few examples

Here’s a list of things that you can do throughout your design cycle to add clarity, and gather insights about your design:

  • You can dig into analytics to try and discover existing insights.
  • You can use Qualaroo, or SquareInsights to ask users questions inside your app.
  • You can chat with happiness/support folks to help determine what the most common complaints, and user pain points are. You could also do support regularly yourself.
  • You can ask other designers for feedback.
  • You can ask your partner/spouse/relative/friend to try your interactive prototype while you watch.
  • You can run some quick usability tests on usertesting.com.
  • You can approach strangers to help you out for 5 min in exchange for a $10 Starbucks gift card.
  • You can ping people in your company that have never seen whatever it is that you’re building, and request 5 minutes of their time, watching them attempt to use your design.
  • You can run a user report in KISSMetrics or MixPanel (or whatever app you use to track users), get a list of actual users, and email them requesting feedback.
  • You can run an A/B test on your design to make sure that it isn’t sinking the ship.

This is by no means an exhaustive list.

Each of these activities will help you validate the hypothesis that’s in your head, and either A) confirm your assumptions, or B) give you insight into how you might tweak your design to make it even better.

Feel free to pick and choose. If you try one of these things a couple of times, and it doesn’t work for you. No worries. The important thing is to try a bunch of them, especially early in your design process. The earlier you incorporate these activities, the quicker you’ll know whether the design you’ve come up with is a great one, or whether it’s a dud.

I’ll repeat this one more time, because I think it’s important:

The sweet spot comes when you A) primarily trust your gut, but also B) use data, and testing to validate your assumptions, and add clarity.

Thoughts? Ping me on Twitter, or shoot me an email: growthdesigner@gmail.com

A big thanks to @hnshah, @melchoyce, and @folletto for reading, and providing feedback for this post.

Rest in peace “growth hacking”

growth-hacking

“Growth Hacking”: Can we lay this term to rest? It just feels… tainted.

To be clear, growth in and of itself isn’t a bad thing. But somehow “growth hacking” has earned itself a bad reputation. Why is that?

To figure this out, let’s journey back to the very beginning.

Doomed by definition

When Sean Ellis coined the term growth hacker in 2010, he stated that:

A growth hacker is a person whose true north is growth.

And therein lies the problem…

Growth for the sake of growth has never been a good LONG-TERM business strategy. Growth for the sake of growth may appear to work wonders in the short-term, but long-term it’s a wolf in sheep’s clothing. Long-term, growth should be centered around people not numbers or percentages.

Everything that a “growth hacker” fights tooth and nail to optimize can be improved across the board by focusing on one thing:

Just focus on making your users lives better

Instead of having your north star be growth, what if instead you focused on the success of your users as your primary objective? Once your primary focus shifts to making your users lives better, everything else that you used to wrestle with as a “growth hacker” will begin to fall into place:

  • Activation – By focusing on your users, and their needs, you’ll have less friction in your new user flow, so more people will stick around.
  • Revenue – With more people sticking around, chances are you’ll make more money.
  • Retention – By definition, if more people stick around, your churn decreases.
  • Referrals – If people find your app remarkable, they’ll spread the word.
  • Acquisition – As people tell others, you’ll acquire more users.

By focusing 100% on users you likely won’t see as many short-term benefits as you would if you continued growth hacking. Unfortunately, one side effect of aggressively optimized short-term growth is that it often comes at the expense of long-term losses. Only, you won’t see the long-term effects for years to come, and by then it’s often too late to do anything about it.

At Automattic our growth explorers are embedded in our design team. Their job is to understand our users (better than anyone else), and to help our users reach their goals, faster (whatever they may be). We’re actively hiring for growth explorers. If this sort of stuff interests you, I’d love to hear from you.

I don’t do comments, but feel free to reach out to me via Twitter or email: growthdesigner@gmail.com

Chef’s table

I found Netflix Chef’s Table series really inspiring. I watched all 6 episode in like 3 days. Here’s the trailer:

I loved how each chef was constantly pushing their limits, constantly learning, constantly creating. There’s no room for stagnation at the top. Not as a chef, not as an artists, not as a designer.

Quote

Slack’s Design

Loved this article about the design of Slack by Andrew Wilkinson of MetaLab:

Some highlights:

We did the logo, the marketing site, and the web and mobile apps, all in just six weeks from start to finish.

I felt the problem had already been solved. It was a crowded market and knew it would be difficult to make his product stand out from the crowd.

Most enterprise software looks like a cheap 70’s prom suit — muted blues and greys everywhere

It Feels Different

Throughout the entire product, everything seems to playfully jump around and pop off the screen.

Like a well-built home, great software focuses on giving its users hundreds of small, satisfying interactions.

In Slack, every piece of copy is seen as an opportunity to be playful.

With Slack, a bubbly, bright UI, delightful interactions, and hilarious copywriting come together to create a personality. A personality which has triggered something powerful in its users: they care about it. They want to share it with others. It feels like a favorite co-worker, not a tool or utility.

Inside Automattic’s remote hiring process

How can Automattic consistently hire the best people without ever having a single voice conversation?

Let’s face it, hiring is tough. When you ask startup founders what their biggest challenge is, hiring is one of the most common answers you’ll hear.

It’s hard enough when you have an office and can interview an applicant in-person. Automattic is 100% distributed. We hire people from all around the world. We can’t meet people in-person to interview them.

You may be wondering… how do you consistently hire great people at a distributed company?

  • Do you fly people out to your location for interviews?
  • Do you schedule phone calls at 4:00am to make it convenient for people applying from Western Australia?
  • How do you know if people will be a good culture fit?

Here’s the inside scoop…

Our hiring process is a bit different than most companies

We don’t schedule chats, we don’t fly people out, and we rarely even have a single voice call before people are hired.

You might think, “That’s crazy, there’s no way that can be effective”.

Well, here are some numbers:

I handle all design and growth hiring at Automattic. I began hiring about a year and a half ago. In that time I’ve reviewed a total of 251 resumes.

63 of those have gone on to an interview (25% of applications received)

41 have been given a trial (65% of those who I interview)

15 have gone on to a final interview (37% of those who do a trial)

14 of those have been hired (93% of those who receive a final interview)

None of those whom I’ve hired have been fired and zero have left the company voluntarily. (100% retention so far)

How do we do it?

I’ve laid out our entire process below in five easy steps:

Step 1) The pre-screen

Ultimately, much of your success with hiring will come down to the degree to which your CEO see’s it as a priority (hint: hiring should be a top priority for every CEO). Matt Mullenweg, Automattic’s founder and CEO consistently spends about 20-30% of his time on hiring. To this day, Matt pre-screens every single resume that comes into Automattic. This takes a significant portion of his time (we get lots of resumes).

“That’s insane”, you might say, “Why invest so much of the CEO’s time just pre-screening resumes? Shouldn’t this be delegated?”

No! Here’s why:

Quality control: By reviewing all applications himself upfront, Matt can ensure from the very beginning that each applicant will be a good fit for the company.
Position throttling: At any point in time, Matt knows better than anyone else which roles in the company have the greatest need for additional resources. By pre-screening all applications he can easily throttle how many applications get forwarded on to each hiring lead, thus effectively increasing or decreasing volume for every role in real time.
Pre-approval: As a hiring lead, knowing that Matt has already pre-approved all resumes that come my way helps me move forward confidently with each applicant. I never have to second-guess as to what Matt might think about a particular applicant, because he’s essentially already pre-approved every application that I see.

Step 2) Second tier resume review

We now have 315 people at Automattic. As such, we have multiple hiring leads for various roles. Every week Matt sends each hiring lead a batch of applications. Mine will consist of design and growth applications exclusively. I’ll then review, and respond to each applicant.

Note: Replying to applicants takes top priority over every other responsibility I have.

If someone doesn’t look like they’d be a good fit, I’ll typically send them the following:

Hi John,

Thanks for your application to Automattic. We don’t think there’s a great fit at this very moment, but I encourage you to keep an eye on the “Work With Us” page on Automattic.com, and also keep us updated and consider re-applying as your skills and contributions to open source projects grow and expand.

Cheers,

Dave Martin

We try to reply to everyone, whether they look like a good fit yet or not. There are good reasons for this. Not only is it the right thing to do, we’ve actually had quite a few Automatticians get hired after reapplying a second or third time.

When someone does look promising, I’ll send the following:

Hi John,

Please add me on Skype (redacted is my username). I’d love to chat with you about the position.

Cheers,

Dave

Typically the applicant will add me on Skype within the next 24hrs.

Step 3) Interview

Immediately after I accept someones Skype invite I’ll kick things off by saying something to the effect of:

Hi John!

Rather than set up a time to chat, let’s just keep a conversation going in Skype. I’ll ask questions, and you can answer as you have time. Could be today, tomorrow, whenever you’re available. Sound good?

Then I just wait for them to respond. By keeping things 100% text, and 100% asynchronous, I don’t have to worry about trying to schedule a day/time to chat (which can definitely be a pain across timezones).

Not only that, the concept of answering questions asynchronously is a relief for most people. Most everyone who applies is currently working elsewhere. Allowing them to answer questions whenever they’re available gives them the freedom to answer before or after work, or even on the weekend.

It also frees me up to work on other things in the meantime. I can have up to five interviews going at any one time without any difficulty, while also working on other stuff in between. It ends up working out really, really well.

After they reply to that first question I’ll let them know that we’ll just keep the chat text-only, and I’ll typically ask an open ended question. Something like:

I’d love to learn more about you. Let’s just keep it as Skype text for now (most of our communication at Automattic is done via text). Tell me a bit more about yourself. What are you passionate about? What do you enjoy doing?

I’m able to glean quite a lot out of this first question. Their answer here shows me how well they communicate. It gives me a glimpse into their interests, which helps determine whether they’ll be a good culture fit at Automattic. It also serves as a bit of an ice-breaker before we dive into questions related to the role.

After we’ve chatted about interests, I’ll dive right into questions related to the job. My goal is to keep interviews as short as possible, while at the same time extracting as much insight as possible. I’m constantly fine tuning the questions that I ask. Here are a couple of examples:

– Can you break down how comfortable are you with the following, on a scale from 1-10 (10 being expert): PHP, JS, react, jQuery, HTML, CSS?
– What one single design that you can share a link or screenshot to are you currently most proud of creating?
– When it comes to growth, would you say you skew more towards A) Doing a little bit of research, and then just starting to test stuff, or B) Devoting more time doing research up front before you narrow in on which tests to run?
– Take a product like WordPress.com. Let’s say you were tasked with increasing engagement (which we define as the number of people who log back in each month). What are some specific ways that you might go about trying to do that?

Every question I ask is strategic. There’s no sense in wasting your time, or theirs asking needless questions. Every question you ask should deliver some level of unique insight. If it doesn’t, you should drop it, or tweak it.

On that note, if half way through the interview you’ve already made up your mind about an applicant, you might as well just end the interview there. When an interview doesn’t work out I’ll typically say something like:

Thanks so much for the interview. I appreciate your taking the time to chat. There were a number of things that I liked about your application, unfortunately I don’t think there’s a great fit at this very moment. I Would encourage you to keep an eye on the “Work With Us” page on Automattic.com, and also keep us updated and consider re-applying as your skills and contributions to open source projects grow and expand. All the best!

If the interview goes well, I’ll ask if they have any questions. Some applicants have lots of questions, some have just a few. The question I get asked most frequently is about next steps, to which I’ll reply:

Everyone that gets hired at Automattic goes through a paid trial process. You are given a project, which you can work on as you have time (could be an hour or two a night, could be on the weekend, whatever works for you). You’ll keep track of the hours you work, and invoice us at the end of your trial. Most people do this while still employed. You can expect the trial to last about a month (but it really depends on how much time you can put in). There is no deadline, it’s done when it’s done. How does that sound?

Some applicants will want to start right away. Some may want to clear up a block of time, and may wish to hold off for a week or two. We’ll happily accommodate whatever works best for them.

Step 4) Trials

Trial projects are essential to effective remote hiring. I can’t imagine hiring people without them.

Every single person who applies to Automattic (no matter the role) does a trial. We pay all trials the same amount ($25/hr).

Trials are a win/win solution for everyone involved.

For applicants:
– A trial gives them a chance to test out whether they’ll enjoy working remotely.
– It gives them an opportunity to take Automattic for a spin, with no strings attached.
– They can stop their trial at any time if they choose.
– Whether they end up getting hired or not, they still get paid for their time.

Trial projects are great for us for a number of reasons:

– They give us insight into how a person actually works.
– We get to see how well a person communicates over the course of a project, and how well they respond to feedback.
– Our trial projects are real projects. They consist of actual work that we need done. In that sense, we’re also getting additional stuff done in the process.

To transition from the interview into the trial itself I’ll usually tell the person:

I’ll be in touch shortly with 2 things:

1) A trial contract (just sign it and email it back)
2) Access to a private blog where we’ll communicate throughout the rest of your trial.

We’ve got an internal tool that I can enter an applicants name, email and position into, which will then send out a trial contract to the applicant.

Once I fire off a contract, I’ll post a comment on our internal hiring P2 to give HR a heads up that a trial contract will be incoming shortly.

NOTE: P2 is a WordPress theme that makes threaded discussions incredibly simple. We use P2 for a large portion of our internal communication.

I’ll then create a new private WordPress.com blog and invite the applicant to be an editor. This is where I’ll post the project brief. This is also where I’ll move the conversation for the remainder of the trial.

“Project brief” you say? “What does that look like?”

Have a look, here’s an example:


Project Brief

Improve the homepage design for automattic.com.

Background

We’ve been meaning to give automattic.com a refresh for some time now. The current design has been there for a number of years. I’d love for you to spend some time coming up with a fresh new look. Just focus on the homepage for this trial project (no need to spend time redesigning any of the other pages).

Resources

  • automattic.com
  • Automattic logo can be found here.
  • I’m happy to hop on a call and go over any of this if you’d like.
  • You can always ping me with questions here whenever you need help, or when you are ready for feedback.
  • Additional internal resources are given here

Let me know if you need access to any additional resources.

Deliverables

Along the way, please post all of your deliverables to this P2.

Round 1: Summary of your thoughts

I’d like for you to first summarize (just a simple bullet list is fine) the problems and opportunities you see with regard to the current website. I’ll review this to make sure we’re on the same page, and then you’ll move on to round 2.

Round 2: Rough concepts

Use whatever method you prefer to convey quick concepts (sketches, wireframes, balsamiq, etc…). Please don’t spend any time on polish at this point. I’m just looking for rough concepts. You’ll then iterate quickly based on feedback until we’re both happy with the direction.

Note: I’m not looking for any sort of re-branding. Feel free to move the logo around, and improve the layout, but no need to redesign the logo for this specific project.

Round 3: Add some polish

Get the design as close to pixel perfect as you can (or optionally hop straight to code, if you’re more comfortable doing that).

Round 4: Code it up

Last, I’ll ask that you code it up. Simple HTML and CSS is fine. Feel free to mix some JS in where appropriate if you feel comfortable doing so (but it’s in no way required).

Your final deliverable should be a zip file with all of your source files, as well as your HTML homepage.

Good luck!

There is no deadline for this project, but I encourage you to communicate early and often (not just when you’ve got something that’s pixel perfect).


I’ll touch base with the person one more time to make sure that they got the contract and access to the private blog, then I’ll just leave it in their hands.

I’ll then work with the person throughout each round of the trial project. We’ll communicate back and forth about the project via the private blog.

If things don’t end up working out, I’ll do my best to highlight why. At this point the applicant has invested quite a bit of time. I try and be as specific as possible as to why they are not going to proceed to a final interview.

If the trial goes well, I’ll chat with them about the final interview with Matt.

Step 5) Final interview

In the final interview Matt will chat with each applicant. This serves as a good opportunity to vet each applicant one last time. This is also where Matt will chat with the person about compensation, and answer any additional questions they may have.

If the final interview goes well, they are sent an offer letter.

Recap & final thoughts

  • As the CEO, you need to take hiring seriously. It should be one of your top priorities. Bookending the hiring process as the first & last touch point is an excellent way to ensure that only the best people get hired.
  • I’d love to be able to give everyone the opportunity to work at Automattic. Telling someone that they aren’t a good fit is hard, and unfortunately it doesn’t seem to get easier with time. The best thing you can do is to try and be helpful and kind to everyone, and to try to get back to people ASAP.
  • Telling people no is hard, but mistakenly bringing on the wrong people can be much worse. While you want to always be kind, and helpful to all applicants, your primary responsibility when hiring is to ensure that only the best people get hired. That is priority number one.
  • Trust your gut. You may often be tempted to “give this person a try” even though deep down you don’t feel right about it. In my experience, this rarely works out, and just ends up wasting a lot of peoples time. Two rules that I’ve learned to live by: A) never interview anyone that I’m not confident will make it to a trial, and B) never offer a trial to anyone who I’m not fairly confident will make it to a final interview.

Interested in learning more about our process? You can read some of Matt’s thoughts in this HBR article.

We’re hiring

We currently have 13 open positions. We’d love to hear from you!

MicroConf 2015

Earlier this week I attended MicroConf for the first time. What a great conference. I admittedly felt like a bit of an imposter (since I’m not actively trying to start my own company), but I still picked up a number of great insights, and met a number of interesting people.

One thing that became quickly obvious is that no one experiments with growth at the speed and efficiency that bootstrap founders experiment with growth. :-) This is of course out of necessity.

A couple of additional thoughts that resonated with me:

Onboarding

  • Duolingo has really stellar onboarding, as does Slack.
  • Close.io, Drip, and Wistia all offer good examples for how to run successful drip email campaigns.

Copy

Loved these insights from @Copyhackers talk:

  • There are very little actual best practices when it comes to copy. DO NOT try and copy and paste from other peoples polished successes – it never works.
  • Headlines and buttons work well when optimized together.
  • Repeat your headline copy in your button.
  • Don’t imply work is involved in button copy.
  • Every element on your page has one job. A headline is meant to keep you on the page. A button is meant to be clicked. Look at each element to see whether they are doing their one job. If not, focus your attention there.
  • In the end, no one has a clue what you should be saying to your customers (nope, not even you!). Just try a bunch of stuff and use whatever works best.
  • While you can’t copy and paste successful copy, you can mine ideas for great copy from testimonials, support emails, reviews, message boards, book reviews on Amazon. Anywhere regular people gather to talk in an un-polished way about your industry.

Sales

Some great insights from @steli:

  • Sales is about you being the most decisive person in the room.
  • Start by qualifying people – what are their needs? People just want to know what’s in it for them.
  • Follow up until you get a yes, or a no. Silence does not equal rejection. Just keep following up.
  • The magic is in the follow up. After 2-3 follow ups most people quit.
  • Keep demos short – 10 min max – focus specifically on how your product will solve their pain, then stop talking.

Pricing

You always hear that you should price your products based on value. When you first launch something, the actual value your product delivers can be quite low. @robwalling talked about “aspirational pricing” where you set a price (that’s potentially too high to start off with), and you keep adding value until the value in your product eventually matches your price.

Overall

What a great community. I loved the sense of inclusion by everyone. I loved that everyone was both a teacher and a student. I loved hearing all of the inside tales about what people are working on, what’s working for them, and what isn’t. In the end, I’d highly recommend this conference.