Developers and Data Scientists: The Enterprise Force Multipliers
"Software is eating the world" is a statement in danger of becoming a cliche, and yet it remains a prescient observation of the way in which every business is becoming a software business. A more important observation is how critical developers are to this transformation for every business function.
Apps have now become the way consumers discover and use services. If your app sucks, your product sucks. Similarly, APIs are the defacto way that both apps and third party companies integrate. If your APIs sucks, your product sucks.
All of this helps explain why Stephen O'Grady describes developers as The New Kingmakers. Yet even in the world of developers, there are three jobs that act as force multipliers within the modern enterprise: native app developers, API developers, and data scientists.
Native app developers
Native apps provide the best application experience because they are optimized for their particular devices. As a result, the demand for native app developers has skyrocketed. My friends in native app development routinely charge over $200 an hour for their services, and they are completely booked into the foreseeable future.
Although a Java developer can probably “get by” and bang out an Android app after a week or two of training, learning how to properly use all the underlying APIs of the operating system takes a long time.
Then there’s attention to detail. Because mobile app screen sizes so small, getting every detail right is important. Unfortunately, 99% of developers don’t have the patience or eye for detail, so even if they know the mobile operating systems inside and out, they simply don’t want that type of job. It explains why very few developers become high-performing native app developers, even though the pay is so good.
API-based systems and microservices architecture (MSA) are reshaping how enterprises build their IT systems. At Apigee, we see API architect and API developer positions at our customers open for months. What makes API developers so rare and hard to find?
Two things: developer experience and understanding scalability. The purpose of building an API is to enable consumption by other developers. Similar to art or UX, there’s no exact formula for making a “good” API. In the late 1990s and the 2000s, there was a huge push by SOA to create APIs formulaically; hence protocols such as SOAP appeared. This was ultimately futile and was abandoned for simpler protocols such as REST.
Unlike UX, there are few developer experience experts in this new world, which means that API developers need to rely on their own intuition and experience writing and consuming code. The job also involves—gasp!—talking and getting feedback from other people (this is possibly a developer’s worst nightmare).
API developers need to be careful not to sacrifice scalability and performance as they make their API easier to use. For inexperienced API developers, this happens a lot. Also, performance and scalability depend on what type of client is calling the API—when developing a mobile API, the developer needs to consider the type of device, and deal with unreliable networks. Very few developers have the experience, enjoy the process of talking to API consumers, and understand of scalability challenges of production systems.
In the future, enterprises that can make sense of the data they collect from customers will be the companies that can better serve their customers. Google, Facebook, and Amazon have all built their enterprises on the data they collect, rewarding their customers and stockholders with amazing gains.
When we think data scientists, some of us of think statistician nerds. But if you look what a data scientist does on a daily basis, it is 20% statistics and 80% data wrangling. Make no mistake, data wrangling is development. Without a solid skillset with languages such as Pig, Python, and SQL, a data scientist relies on other developers for data wrangling, and this slows the pace at which insights are generated.
It’s no wonder that Google and LinkedIn hire aggressively out of graduate machine learning programs in computer science—it's the only program that trains up both coding and statistical skills.
So if tech giants have cornered the market on machine learning developers, where do enterprises hire data scientists? From the pool of graduate physicists! Ph.D. physicists suffer high unemployment, though they are likely to have built a strong skill-set in data wrangling. For example, at CERN, which operates the world’s largest super collider, physicists often data wrangle 75 new petabytes of data per year.
It takes a “village” of different developers to make a digital business successful. However, enterprises stand to benefit greatly if they pay special attention to hiring and training these key developer roles.
Photo: Les Chatfield/Flickr