Engineering Across Platforms

Posted on 5/06/2013 by Erik Schultink, CTO

In the last year or so, we’ve been dealing with a big problem that many web 2.0 companies have also faced: Going mobile.

I’ve benefited from reading others’ thoughts on the subject, but we’ve learned quite a bit through our own experiences. I’ll describe where we’ve come from and the six big changes we’re making to be a more cross-platform company, from the perspective of engineering management. Hopefully you’ll find some of these lessons helpful for your team.

Web was our DNA
We were birthed in AJAX; our desktop web client was our flagship, in which we sought to produce a fast and rich user experience. It was built to leverage the most compelling techniques that Javascript, in the latest web browsers, could provide. Our JS experts were the rockstars in the company, prized by our recruiters. While we also built a simple, mobile version of the site, our intent was merely to give users a way to stay connected when they couldn’t access a “real” browser. I don’t believe we were alone in this approach..

In 2007-08, mobile was niche; it was utilitarian. Tablets were clumsy novelty toys. iPhones couldn’t run apps. There was something called WAP. Android didn’t exist. We saw mobile use cases like “entertain yourself on the bus”, “pull up the details about the event while you’re on the way”, “get notified when you’re tagged in a photo (with option to untag)”. It was basic - get stuff done, keep in touch - when no better alternative existed.

What we were wrong about was how quickly mobile app platforms would become “good enough”. Not just good enough for those basic utility cases - but for just about everything. Sure a full size screen might give you larger photos and a keyboard might help you write messages more quickly - but people were more willing to accept slightly smaller photos and slower typing than we anticipated.

Where this left us, in 2010, was with an organization of about 150 people in development-related roles (product, design, qa, engineering, etc) geared towards our flagship JavaScript-based client. We had Android, iOS, J2ME, and Blackberry development ongoing - but these were not where the balance of our people or attention lay. This neglect showed in the tools and processes we had to support development on these platforms.

What we had to change
I’ll go through each in detail, but let me summarize the changes that we’re making to transform our engineering organization for this challenge:
1. Architecture of clients and services
2. Interdisciplinary, product-oriented project teams
3. “Core” client/framework teams that build base components and support project teams with expert-level advising and code review
4. Cross-platform training program
5. More code review

Architecture of Clients and Services
We started as a big monolithic web app. That legacy remains. What you experience on and is, in many places, a presentation/client layer that is quite tightly coupled to the backend data access/management layer that powers it. It probably will be for a long time.

These backend layers - and the communication between those layers and the html/javascript “client” - were built first and foremost for a web context. The assumption is that our client is running in a user’s modern browser, fully capable of making multiple, parallel AJAX calls to our servers over a network that is usually fast and reliable. Data structures, abstraction, logic, etc are all designed accordingly. Our mobile clients - where this assumption does not hold as well - were a special-case, communicating with our data layer via an API that is, in truth, something of a client itself. 

To improve things, we’re moving towards a more a more service-oriented architecture (SOA). Each backend component is a service, exposing a well-defined API. Those APIs are designed not just for “web” clients that happen to run in the same PHP process but also remote clients. To the extent we have any kind of mobile-client-specific “API” layer, it functions as an adapter, attempting to be agnostic of both the particular service and client between which it’s mediating. 

As we live in the real world, and have a massive set of products and services already running, most of our stuff doesn’t comply with this vision. But that’s where we’re moving.

Team Structure
We learned long ago that the best way to scale a product dev org was to have inter-disciplinary teams. The idea is that for all the typical work that might come the team’s way, they have, internally, someone with the skills to get it done. They probably don’t have “the best” person in the company to do a given task; that’s OK. The point is that they are empowered to execute within their area of product responsibility, with minimal external dependencies.

For web, this mostly worked. We had teams with frontend engineers, backend engineers, and usually a designer, QA, and PM. We eventually even eliminated the frontend vs backend distinction to encourage people to be generalists. While I acknowledge that in practice people tend to get tracked one way or the other, the principle is that everyone should have the knowledge to work up and down the stack. For the extreme expert work, we had some core framework teams (next section) that built core bits and could offer guidance to help people working on a product team, if they encountered something particularly esoteric.

We’re extending this same structure across our mobile applications. For each feature that a team owns, they own that feature in each of client applications. Therefore, to be empowered to execute independently, they must have someone with the capability to build for that client. We staff accordingly, trying to spread people with not only php/js knowledge across our teams, but with android and iOS knowledge as well. As we recognize that work for a specific platform will vary for a team overtime, we want our people to be generalists, with the knowledge to work across more than one platform and we have created a training program (described below) to try to expand this.  

Core Framework Teams
For a number of years, we’ve had core framework and infrastructure teams for the web. As with our project team model, we generalized this for other platforms by creating, for each client platform we build for, a “core” team to own that platform within Tuenti. 

These core teams aren’t the main people building features for the platform. Rather, they build the base architecture and shared components, maintain a standard developer tool set for the platform, and define best practices and code standards. Operationally, they advise teams building for the platform, review their code, and manage the final stages of the integration cycle for clients on the platform. 

For our service platform (based in PHP), we similarly have a “backend platform” team that owns this layer, and performs analogous functions for services as the other teams do for clients.

We were mostly web guys (and girls). While we firmly hold the belief that programming languages are mere tools - and a good engineer chooses the appropriate tool for the situation - we certainly recognize that building for web vs mobile is about more than syntax and language features. There are differences in paradigms: concurrency, memory management, connectivity, etc. While most of us touched on these things back in university, a lot of these issues were safely ignored in day-to-day web work. 

To change, we’ve introduced a cross-platform training programming to step people through building their competency in a new platform.
- Level 0 “Exposed” - we start people with self-guided reading and “get to know you” talks, to familiarize people with the platform.  This stage is aimed at giving you a flavor of the platform and background on mobile concepts, in preparation for more formal training.
- Level 1 “Set-up” is to give you a full, working environment of all our tools for the platform, a repo clone, and a very basic knowledge of our specific architecture. The idea is that, in principle, if told to fix a typo on screen 3 of the registration flow in the app, you could, after digging around for no more than an hour or two, get this done and commit your change back. We offer workshops led by experienced engineers to walk people through this.
- Level 2 “Capable” means you have sufficient understanding of the frameworks, app architecture, and tests to code a small changeset (bugfix, feature tweak) that will pass review. Engineers are assigned a mentor to help them, should they need any support in this.
- Levels 3+ “Comfortable” - at this point, you can do day-to-day work independently.  From there, it’s just increasing knowledge/comfort/productivity with the platform - tracked only for the purpose of balancing skills across teams to match the features they’re building.

More Code Review
We’ve practiced code review for a long time. However, I would say that it used to be more of an encouraged best practice than an institutionalized (required) step in our dev process. That’s had to change for several reasons:
- First, while we want to empower anybody to work on any client/service, we can’t have a free-for-all; a specific team still owns each component and needs awareness/approval over what’s being done (and how).
- Second, it’s a natural, organic way to train people, share knowledge, and have ongoing feedback flow between peers. These first two points mattered on web, but they matter even more when you multiply the clients each team must now worry about and the platforms on which a given engineer must develop.
- Third, compared to our web apps, our mobile integration process is heavier and we ship less often, which means the cost of a bad change reaching either 1) integration, or 2) production is much higher (as we can’t discover and fix/revert it in hours or less). I’ll describe the integration/release process for mobile a bit more below.

To become more rigorous and embed code review in our process, we’re becoming more disciplined in our use of JIRA to track work and Crucible to review code. 

Like any organization, we’re still a work in progress. Changing the expectations and culture of 150 people is hard.  While the target vision will undoubtedly change before we’re 100% there, I’ve tried to give you an idea of how we’re facing a more cross-platform world. 

I might follow up with some more details on specific topics. If you have any specific questions that you’d like me to address, feel free to ask them via Twitter or Quora.

Follow us