Last day(s) to participate for a chance to win a FREE space at our Data Science Boot Camp

A free place on our pioneering Data Science Boot Camp training programme is being offered by specialist recruitment agency, MBN Solutions. Places on the much-anticipated course, aimed at upskilling those with raw analytical grounding into bona fide data scientists, are worth £7,000. The average cost of recruiting a data science specialist is £15,000.

The Data Lab has partnered with New York’s globally renowned, The Data Incubator (whose courses are reputedly harder to get into than Harvard), to develop the three-week data Boot Camp as part of a drive to plug the nation’s data skills gap. It is aimed at helping to unlock the economic potential of data to Scotland, estimated to be worth £17 billion* in Scotland alone.

To apply for the MBN Solutions sponsored place, potential participants need to submit a video explaining how they would use the data science Boot Camp training in their current organisations. The video should be maximum two minutes and include:

• Your current role and experience
• Why you want to take part in the course
• Why you believe improving your skills in data science is important
• How you hope to use the skills you will learn in the course to improve your work 
• What impact do you expect to achieve for your organisation as a result of your skills

The video must be uploaded to YouTube, the link to the video sent to skills@thedatalab.com by 12th August.

Michael Young, CEO of MBN Solutions, said: “With the average cost of recruiting a data scientist £15,000, the Boot Camp presents an incredible opportunity to upskill current staff and invest in your company’s data science offering.

“The Data Incubator is recognised as the go-to experts in the data training sector globally and, by sponsoring a place for a budding data scientist, we are helping to enhance Scotland’s pipeline of data science talent.

“Every day we see fantastic, innovative data science projects going on in our client’s organisations, Scotland is leading the way in data science in the UK and The Data Lab are really driving the data agenda forward. Some countries are only just waking up to the potential of data. This course marks a really exciting time for Scotland and The Data Lab and we at MBN Solutions are thrilled to be a part of it.”

Brian Hills, Head of Data at The Data Lab, said: “We’re very pleased to have MBN Solutions sponsor a place on the Boot Camp which will take us one step closer to exploiting the data opportunity in great demand and short supply.

“It is going to be an incredible three weeks with attendees gaining a highly sought after data science skillset and learnings from world-leaders in data science.

“It’s crucial Scotland remains ahead of the curve in data science. By investing in our pipeline of talent and learning from international experts, we are securing our future and taking critical steps toward exploiting the data potential available here in Scotland.”

The pioneering training initiative will allow Scottish businesses to fast track potential returns by using data analysis to drive insight and decision-making across industry. There are only a few places left for the Boot Camp which will take place in September in Edinburgh. It will focus on developing practical application skills such as advanced python, machine learning and data visualisation in a collaborative environment.

For further information on the Boot Camp, how to apply, and how to enter the competition, please check out our Boot camp page, download our brochure or email skills@thedatalab.com

About The Data Incubator

The Data Incubator is data science education company based in NYC, DC, and SF with both corporate training and hiring offerings. They leverage real world business cases to offer customized, in-house training solutions in data and analytics. They also offer partners the opportunity to hire from their 8 week fellowship training PhDs to become data scientists. The fellowship selects 2% of its 2000+ quarterly applicants and is free for fellows. Hiring companies (including EBay, Capital One, AIG, and Genentech) pay a recruiting fee only if they successfully hire. You can read more about The Data Incubator on Harvard Business Review, VentureBeat, or The Next Web, or read about their alumni at Palantir or the NYTimes.

About MBN Solutions

In a field saturated by many lookalike recruitment consultancies, MBN is a truly different business. Priding ourselves on values of deep, real subject matter knowledge in the Data Science, Big Data, Analytics and Technology space, a passionate approach to developing our own consultants and a strategy placing our clients at the heart of our business, MBN are a true market defining ‘People Solutions’ business.

The big data challenge: Extracting actual business value

You've got the tools and the power of the cloud to capture big data, but figuring out what you want from it and how to extract it is the final, crucial challenge.

Advances in data networks and storage mean organizations capture far more data than they ever have - perhaps a stream of measurements from manufacturing equipment, from vehicles, or from game-changers like web-enabled refrigerators (no, I've never seen one either).

The enterprise CTO may have the data storage part all figured out - theirMongoDB cloud database is in place, or they rent DBaaS from Cloudant. But why? What does an enterprise do with all this unstructured data?

The first thing is to identify what the enterprise wants. Analytics can be an area of blind faith – if the enterprise is not clear about its big data needs, it may just hope that something good pops out.
Identify the big data needs.

Big data analytics, like all IT, is subordinate to business needs. An organization must figure out their requirements before working on big data.

No two organizations are the same, so there is always a variation in needs. The IT department may receive requirements like these.

Crunch data for instant reports.
Decode telemetry on the fly.
Find a needle in a haystack in a vast quantity of signals.
Find the regular operational patterns in a vast quantity of signals.

Analytics is a service-oriented area so the CTO could just finish his work there and outsource the rest. If he decides to keep it in-house, he needs a few more things.
Get some analytics applications.

Analytics applications help turn large data sets into business value. The enterprise uses analytics tools to tackle the difficult job of doing something useful with their unstructured data.

Data analytics products are one of the big data technologies and live in a data scientist's toolbox. Analytics products don't usually deliver ready-made business value.

When an organization purchases analytics applications, they must leave plenty of cash for the training budget. Complex tools are not intuitive.
Write a big data policy.
Managing large data sets is a difficult job. The big data manager has plenty of moving parts to configure to meet these requirements.

What is the retention policy? What parts of the data pool can be deleted, and when? What happens to the rest of the historical data?
What is the data protection policy? Who gets to view data? What are the privacy implications? What are the legal restrictions?
Where is the data stored? If a cloud provider is holding the data, how do we get it back?
What kind of meta-data is required? How can anyone identify the purpose of a big data store?
How many data sets are there, and how can they be blended?
Assemble an analysis team.
The first part of building a team is partnering up a business executive and an IT sponsor. Both are required.
There may be a data warehouse and data miners in the organization, but probably no data scientists. There are a few ways of getting some.
Hire experts. Pros are in demand.
Hire people with the right capability and let them learn.
Spot the budding statisticians in your organization and grab them.

Spotting capability means looking for clues. John Foreman is chief scientist at Mailchimp and writes a blog on data science. If someone is a fan of his work, that's a clue. Perhaps one of the data miners has an artistic streak. The person obsessively dragging consumer behaviour out of click trails is worth talking to.
That still leaves some gaps.

A few huge organizations, like telecoms companies and global retailers, have been battling with the problem of analytics for decades. They have specialist teams, home-grown tools, and years of experience. Alongside their expensive specialized capabilities, a brave new world of big data and commoditized data analytics is appearing. There is quite a way to go.

The enterprise is doing new things with existing data sets, rather than collecting new data.
Plenty of big data tools exist, but few tools ready for business users.
Organizations in many parts of the world have not started exploiting big data.
Better machine learning is required to extract signal from noise.

It takes statistical, technical and business expertise to get value from big data. Even where the analytics tools exist, they must be tailored for business needs - it's not a one-size-fits-all world.

Over to you, big data startups around the world. Plug those gaps

Industry Speaks: Top 33 Big Data Predictions for 2016

By Alex Woodie

What will happen in big data in 2016? You’d think that would be a cinch to answer, what with all the deep neural net and prescriptive analytic progress being made these days. But in fact the big data predictions from the industry are all over the map.Datanami received dozens of predictions from prominent players in the industry. Here is a culled collection of the most interesting ones.

Oracle sees the rise of a new type of user: the Data Civilian. “While complex statistics may still be limited to data scientists, data-driven decision-making shouldn’t be,” Big Red says. “In the coming year, simpler big data discovery tools will let business analysts shop for datasets in enterprise Hadoop clusters, reshape them into new mashup combinations, and even analyze them with exploratory machine learning techniques.”

Nucleus Research is going out on a limb and predicting the death of big data as we know it. “In the past two years everyone and their dog seems to have launched a big data solution of some kind. It’s time for the shiny object syndrome to stop,” it says. “Instead of attacking the monolithic and daunting task of big data analysis, users will approach and access it like any data.”

Since even canines are pulling down big VC money for data startups, it may be time to start asking tough questions, according to Keri Smith, senior vice president at Opera Solutions. “What is the real ROI of a big data solution?” Smith asks. “How can companies get beyond departmental deployments to maximize the value of big data across the enterprise? And what are the meaningful use cases across a variety of verticals? If your company isn’t asking these questions and actively seeking answers, it should soon.”

We’ll see the rise of Data Jedis in 2016, says Matt Bencke, CEO of Spare5. “More jobs will be changed by AI than ever before and the ‘Data Jedis’ will become the most sought after employees,” he writes. “Machine learning+human insights will infiltrate new industries including healthcare and security and employees will need to adapt to providing a different service or get left behind in 2015.”

Data science will be big in banking, predicts Mike Weston, CEO of data science consultancy Profusion. “The financial industry is one of the pioneers of data science techniques,” he writes. “Nevertheless, the adoption of data science has been far from uniform across all banking services. In 2016 I expect this picture to change. Better use of data and personalisation of services will move from the financial markets to retail banking. It will have a profound impact on marketing, customer service and product development.”

The prospect of advanced AI giving rise to robot overlords scares Elon Musk. But according to Jans Aasman, a cognitive scientist and CEO ofFranz, AI should be placed the “friendlies” column. “Artificial intelligence and cognitive computing will make personalized medicine a reality, help save the lives of people with rare diseases and improve the overall state of healthcare in 2016 and beyond,” he says.

Chief Data Officers (CDOs) will become the “new it girl” of information tech, complicating office politics forever, argues Michael Ludwig, head of Blazent’s Office of the CTO. “Driven by the complexity of big data and the need for complete and accurate data, the CDO will become increasingly important,” he writes. “As a result, the CTO and CIO will need to make room for the CDO, and tension will emerge within the C-suite until clearly defined roles and associated teams are established.”

Not everybody sees it that way, including Craig Zawada, Chief Visionary Officer at PROS. “In 2016, we’ll begin to see erosion in the appointment of Chief Data Officers, a role of the past. Instead, Chief Insight Officers will emerge in 2016 as crucial leaders in the big data compilation process.”

CIOs, yeah baby!

But can the mighty CIO get his mojo back? Cazena founder and CEO Prat Moghe’s looks into his crystal ball, and says it’s so. “In 2016, CIOs will take advantage of enterprise-ready cloud services to become brokers of cloud services that meet IT mandates for governance, compliance and security as well as business needs for agility and responsiveness,” he writes.

Streaming analytics will start to mature and prove its worth in the big data lineup, predicts Phu Hoang, the CEO and co-founder ofDataTorrent. “While lots of companies have already accepted that real-time streaming is valuable, we’ll see users looking to take it one step further to quantify their streaming use cases. In the next year, customers using streaming tools will reach new levels of sophistication and demand a quantified ROI for streaming analytics,” he says.

Real-time analytics will be hot next year. We get it. But one technology—Apache Kafka–stands taller than the rest, according to MongoDB‘s VP of strategy Kelly Stirman. “Kafka will become an essential integration point in enterprise data infrastructure, facilitating the creation of intelligent, distributed systems,” Stirman writes. “Kafka and other streaming systems like Spark and Storm will complement databases as critical pieces of the enterprise stack for managing data across applications and data centers.”

Like drums? Then you’re going to love 2016, says Badri Raghavan, the chief data scientist at FirstFuel Software. “In the months ahead, we will see organizations and individuals tap data and analytics to deliver personalized and engaging experiences across industries including energy, sports, social good and music. For instance, people will be able to use data to change a song based on their personal preferences (e.g., lots of drum).”

How will the IoT impact the semiconductor business? IT legend Ray Zinn has a few thoughts on that. “You will see greater divisions between design and fabrication,” he writes. “Fabs will have the mission of scale to serve a few billions consumers and the nascent Internet of Things (IoT) markets. Design will become uniquely divorced from fabrication, splitting the market risk. Design firms will survive best by innovation, and fabs through ruthless efficiency. The question is what comes next? There will inevitably be new markets and devices that will drive a new growth spurt. The IoT is the sleeping giant, but I doubt the only one snoozing.”

Machine learning, big data automation, and artificial intelligence were big in 2015, and will get bigger next year, says Abdul Razack, SVP & head of platforms, big data and analytics at Infosys. “In 2016, the pace at which enterprises more widely adopt artificial intelligence to replace manual, repetitive tasks will rapidly increase,” Razack says, citing the $1 billion AI investmentmade recently by Toyota. Big data automation is already growing, but next year “it will be more widely used to accentuate the unique human ability to take complex problems and deliver creative solutions to them.” The self-driving cars from Tesla have built-in machine learning, but next year, “machine learning will quietly find its way into the household, making the objects around us not just connected.”

Lots of people see exciting things happening in the big data space in 2016. Not Charles Caldwell, the vice president of solutions engineering and services at Logi Analytics. “When I look ahead to 2016, I don’t see a lot of exciting things happening. Other vendors have come out with their predictions around cloud, visual analytics and mobile, but most of those things are old trends that are settling down. In my opinion, 2016 will be a year of consolidation and ground building for the next big thing.”

The “Not In Your Wildest Dreams” award goes to Peter Eicher, senior manager for product marketing at Catalogic Software. We’re not talking about his prediction that copy data management (CDM) “is a technology whose time has come as evidenced not only by the new vendors in the space but by old school players chiming in with ‘me too’ arguments.” That makes total sense. No, we’re calling Peter out for his crazy prediction that the New York Knicks win the NBA Championship. “Yeah, not happening,” he admits. “I can’t be right all the time. On the other hand, that prediction has been wrong for 42 years running. One of these days….”

The “Debbie Downer” award for big data goes to BlueTalonCEO Eric Tilenius for his prediction that the pace of big data breaches at major enterprises may rise. “In 2016, the lack of unified data governance could lead to the biggest security disruption that enterprises have ever faced—comparable to the disruption caused to the traditional enterprise perimeter by the entry of mobile,” he writes. “Relying on a fragmented approach to control data access, where inconsistent policies are applied across an ever-changing data landscape, will leave gaping holes in the protection of enterprise data.”

Are you into microservices? If not, you will be soon, according to SaaS heavy Workday. “It’s clear that the on-premise versus cloud battle is over. Cloud has won,” the company says. “Yet, not all cloud architectures will be created equal. Microservices architectures will go beyond the realm of consumer Internet designs like Netflix and become the most important architecture advancement in enterprise applications since the shift to the cloud.”

Big data is hard, and companies will struggle with it next year, says Ulrik Pederson, CTO of TARGIT. “2016 will see an expansion of big data analytics with tools that make it possible for business users to perform comprehensive self-service exploration with big data when they need it, without major hand holding from IT,” he writes. “Corresponding with my first predication, I anticipate a huge increase in advanced analytics projects across industries. However, that doesn’t mean they’ll be successful…. I wouldn’t be surprised to hear of many vendors and customers struggling to implement successfully.”

The International Institute of Analytics sees the rise of analytics microservices to facilitate embedded analytics. The IIA also sees progress being made in areas of cognitive technology, data science, and data curation. Oh, and the analytics talent crutch will ease as many new university program come online, the group says.

Elnur/Shutterstock.com

People who aren’t data geeks will get into the big data swing of things, says Bruno Aziza, Chief Marketing Officer of OLAP-on-Hadoop provider AtScale. “As Hadoop becomes more accessible to non-data geeks, marketers will begin to access more data for better decision making,” he writes. “Hadoop’s deeper and wider view of data will enable marketers to capture behaviors leading to decisions and understand the processes underlying customer journeys.”

We’ll see more HPC tech making its way into the mainstream, particularly as it pertains to storage, predicts storage giant DDN. “Storage, data management and application acceleration technologies from the HPC industry will continue being tapped at even a higher rate in 2016 to meet the evolving requirements of performance and scale and will replace traditional IT infrastructures at even a higher rate,” the company says.

Impressed with open source big data tech? You haven’t seen anything yet, says Pentaho CEO Quentin Gallivan. “The explosion of cool new tools like Spark, Docker, Kafka, Solr–emerging open source tools designed to enable large-scale, high-volume analytics on petabytes of data are moving from the ‘awkward teenager’ phase to the ‘bearded hipster’ phase,” Gallivan writes.

Spark will kill MapReduce, but save Hadoop, says Monte Zweben, co-founder and CEO of RDBMS-on-Hadoop vendor Splice Machine. “MapReduce is quite esoteric. Its slow, batch nature and high level of complexity can make it unattractive for many enterprises,” he writes. “Spark, because of its speed, is much more natural, mathematical, and convenient for programmers. Spark will reinvigorate Hadoop, and in 2016, nine out of every 10 projects on Hadoop will be Spark-related projects.”

But that doesn’t mean every Spark project will involve Hadoop, says Bob Muglia, the CEO ofSnowflake Computing. “Today, Spark is part of Hadoop distributions and is widely associated with Hadoop. Expect to see that change in 2016 as Spark goes its own way, establishing a separate, vibrant ecosystem. In fact, you can expect to see the major cloud vendors release their own Spark PaaS offerings. Will we see an Elastic Spark? Good chance.”

Organizations will reset on Apache Hadoop, says Dan Graham, general manager of enterprise systems atTeradata. “As Hadoop and related open source technologies move beyond knowledge gathering and the hype abates, enterprises will hit the reset button on (not abandon) their Hadoop deployments to address lessons learned – particularly around governance, data integration, security, and reliability.

The junk drawer problem is one of the Hadoop community’s biggest challenges. But never fear–Master Data Man(agement) is here! “MDM will become ubiquitous,” writes Manish Sood, CEO and founder of Reltio. “MDM as a discipline has long only been affordable by large companies with big IT teams and budget for hardware, software and multi-year implementation projects…A new breed of data-driven applications will come built-in with MDM as table stakes. As a consequence of delivering both operational and analytical functionality, the reliable data foundation of each application is powered by an MDM engine.”

Hadoop will be at a crossroads in 2016, but which fork will it take? Mike Maciag, COO ofAltiscale, give us his prediction. “In 2016, we will see industry standards for Hadoop solidify. In the beginning of 2015, we saw the launch of the Open Data Platform Initiative (ODPi), which established standards for how key projects in the Big Data ecosystem can work together. ODPi doubled in membership during the course of the year as the benefits to standardization for customers became even more clear. We expect to see more growth and recognition in 2016, allowing new technologies and applications to meet the Hadoop ecosystem standards being established by the ODPi.”

We’ll see the emergence of IoT 2.0 predicts Zebra Technologies. “The IoT market will transition to more mature, industry and adaptable solutions from what used to be closed, proprietary first-generation offerings. With an open-source approach, organizations will be able to choose from a larger pool of service providers and their respective APIs.”

The IoT may hearken the rise of a post-scarcity economy, predicts OpenText CEO Mark Barrenechea. “Imagine algorithms as apps for applying big data analysis over the connected masses of information generated by the IoT and its billions upon billions of connected devices in every aspect of our lives,” he writes. “Owning the data, analyzing the data, and improving and innovating become the keys to corporate success—all empowered by a connected digital society.”

The rise of converged platforms that can handle both analytic and transactional workloads will take a leap forward, foresees John Schroeder, CEO of MapR Technologies. “In 2016, we will see converged approaches become mainstream as leading companies reap the benefits of combining production workloads with analytics to adjust quickly to changing customer preferences, competitive pressures, and business conditions. This convergence speeds the ‘data to action’ cycle for organizations and removes the time lag between analytics and business impact.”

Another proponent of a single stack emerging in 2016 is Stefan Groschupf, the CEO ofDatameer. “When a technology category is new, various companies emerge with individual products that aim to provide a solution for a portion of the space,” he writes. “This leaves customers buying a number of tools and trying to learn how to use them together. Eventually, that just won’t do, and customers tend towards an integrated stack of products – or a widely-scoped product – from a single vendor. 2016 will mark the beginning of that transition for big data products.”

Outsourcing will be big in 2016, predicts Anil Kaul, CEO of big data service provider Absolutdata. “A gigantic amount of valuable information can be generated from big data, but accessing this could be challenging and it typically lies beyond the scope of routine business intelligence,” he writes. “Many companies today are partnering with third parties to create and execute big data analytics strategies. Integrating external experts into the big data team may be the best way for companies to stay ahead in this quickly evolving space.”