Inspired? Join our team

Are you a software engineer looking to get into the world of AI and Machine learning? We're looking for a collaborative Data engineer with a passion for working with data and delivering resilient data products rapidly in the cloud.

Experience working in Azure is a must.

White

Senior Data Engineer

Jobs

Welcome to Datasparq’s technology blog. At Datasparq, we design, deliver and run bespoke data science products to help organisations capture value from data. - Medium

Datasparq Technology - Medium

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6PhIRwYf86sTsQg5"><figcaption>Photo by <a href="https://unsplash.com/@popnzebra?utm_source=medium&amp;utm_medium=referral">Pop &amp; Zebra</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure>Imagine a large web portal is reaching out to you to create an AI solution that will be recommending the most relevant articles to users. That’s exactly the task we have been tackling in Datasparq recently. We needed to determine which articles had multiple parts, and what order they should go in based on the titles alone so that the next article in the sequence can always be recommended.
But articles are not the only example where you can come across sequential data. Consider movies and TV series for instance — they would not have produced Avatar 2, 3, 4 and 5 if the first film had not been a success. In this article, we are going to replicate the solution we devised and create a BigQuery table of movie sequels so that we know which movie follows the preceding one e.g. Avatar 2 comes after Avatar and Star Wars: The Rise of Skywalker is the continuation of Star Wars: The Last Jedi.
We use “IMDb movies extensive dataset” available on Kaggle: <a href="https://www.kaggle.com/stefanoleone992/imdb-extensive-dataset">https://www.kaggle.com/stefanoleone992/imdb-extensive-dataset</a>, which contains over 85,000 movie titles. This massive dataset comprises four smaller chunks, namely IMDB movies.csv, IMDB names.csv, IMDB ratings.csv, and IMDB title_principals.csv. We are interested in the latter because this is the one that contains the titles that we will parse and from which we will extract sequences. Having said that, let’s get started!
<figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*kpWnI2NCK2K6CIKsp7WkZA.png"><figcaption>How would you know these two movies are related if there was not any text on the posters?</figcaption></figure><h3>Setting things up</h3>
To analyse this dataset, we will be using BigQuery which is one of the key GCP tools. Although this is a paid service, everybody is eligible for $300 credit to explore this massive toolset (which is way more than enough for the purposes of this project).
To load the data, we first need to create a dataset in which we insert the table. Specifying the location, choose the one which is the closest to you, so that data latency will be the lowest, and queries will be executed faster. We will be creating a table from the .csv containing all data. For convenience, it is best to upload this file into the dedicated bucket on Google Cloud Storage (GCS).
Letting it figure out the schema automatically will probably raise an error because some of the columns do not have consistent data. The (not-ideal) solution we are going to apply now is to set these columns with the type of STRING. To save your time defining everything manually, just copy and paste the following to Schema as text.
<pre>imdb_title_id:STRING, title:STRING, original_title:STRING, year:STRING, date_published:STRING, genre:STRING, duration:INTEGER, country:STRING, language:STRING, director:STRING, writer:STRING, production_company:STRING, actors:STRING, description:STRING, avg_vote:FLOAT, votes:INTEGER, budget:STRING, usa_gross_income:STRING, worlwide_gross_income:STRING, metascore:STRING, reviews_from_users:FLOAT, reviews_from_critics:FLOAT</pre>
<h3>Levenshtein distance</h3>
Now we are moving to the cornerstone of this article — Levenshtein distance. In the simplest words, this metric tells us how much one text string is different from another i.e. how many characters do we need to change in one string to make it identical to the other.
Unfortunately, there is no ready-made Levenshtein distance function to use in BigQuery so we need to create one on our own. BigQuery supports creating user-defined functions (UDFs) written as SQL expression or JavaScript code. Luckily for us, Andrei wrote the Levenshtein distance function a couple of years back and published it on GitHub under the MIT License: <a href="https://gist.github.com/andrei-m/982927">https://gist.github.com/andrei-m/982927</a>. We are going to use his bit of code. However — we will implement a few changes.
Currently the code counts the exact Levenshtein distance, regardless of how big it could be. Considering the huge amount of records present within the dataset this would be a very computationally heavy process involving lots of time to run the whole query. Moreover, this would consume a much larger amount of resource which would consequently lead to higher bills to pay for GCP (or to exhausting the free $300 credit much quicker).
The thing is, we do not really care about the exact Levenshtein distance number because as soon as it crosses the determined boundary, it is beyond our limit anyway. To address that, and significantly speed up the query execution, we are going to return the max set boundary for the given title, as soon as it is crossed. Implementation of these changes will allow us to execute the code approx 100x faster!
<a href="https://medium.com/media/da854d34ed454d83b93ec2b0b357716f/href">https://medium.com/media/da854d34ed454d83b93ec2b0b357716f/href</a><h3>Data cleansing</h3>
To speed up performance and increase readability, we don’t want to select all the columns from the dataset but rather focus on these (which can be useful in the later stages). The most appropriate ones in our case will be imdb_title_id, title, director, and date_published.
After a quick dataset exploration, we realise the sequence is expressed in several ways. Below are a few examples.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/266/1*3RAoZz9cuYSRPDa82_B_PA.png"><figcaption>Figure 1 — sequence at the end of the title</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/278/1*Y75i5c1erGCNyPGUXljhCw.png"><figcaption>Figure 2 — sequence written as a roman numeral</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*rjPwvebCFNlKFq61RIXjQQ.png"><figcaption>Figure 3 — sequence in the middle</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/582/1*oL2dh7YV81rLTp6yKET6GQ.png"><figcaption>Figure 4 — sequence at the beginning of the title</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*YEdKAHzrOPe5h2ZrqyIRaA.png"><figcaption>Figure 5 — sequence as the year at the end</figcaption></figure>Data engineering involves a lot of customisation of the data we work with. To extract all sequences from the various titles, we will need to adjust our query to catch all of the cases.
As was already mentioned, Levenshtein distance checks how many characters we need to change in one string to make it identical to the other one. What we strive for, is to find the golden mean — in terms of how many characters we allow to be different without having a negative impact on our final result. We do not want to set it too high, as we do not expect the next sequel to have a totally different title. It’s just the sequence that will be different. We also want to minimise the number of comparisons, to avoid extra costs associated with the query processing and to make it run faster.
And here comes the problem… If we do not set the distance high enough, we will miss plenty of movies that are actually sequential e.g. those in Figure 3, which list the subtitle as well as the title. Fortunately, there is a simple workaround we can use in that situation. We just have to REPLACE() these substrings with empty strings so taking Figure 3 as an example; “Destination Dubai” will be deleted leaving just “The Wedding Party 2”. The solution works perfectly fine, removing what is unnecessary and just keeping the string with a valuable number.
<a href="https://medium.com/media/302df79c6198e582e304122992159196/href">https://medium.com/media/302df79c6198e582e304122992159196/href</a><h4>Roman numerals to Arabic numbers:</h4>
Here comes another tricky bit. We cannot just catch the sequence written as a Roman numeral and REPLACE()it with the Arabic one because this will only pass muster in a very simple situation e.g. when the roman numeral is one of I, V, X, etc. What can we do then? Well, this is another time we can make use of UDFs and create such a method that will locate the Roman numeral, find an Arabic representation of it, and finally replace it. That is exactly what Mikhail did in this StackOverflow post:
<a href="https://stackoverflow.com/questions/69771597/what-is-the-best-way-of-converting-roman-numerals-to-arabic-ones-in-bigquery">What is the best way of converting roman numerals to arabic ones in bigquery?</a>
After a little modification of Mikhail’s code to suit our needs, this is what we are going to use:
<a href="https://medium.com/media/f43ce234a2ce501b056ca185896b3400/href">https://medium.com/media/f43ce234a2ce501b056ca185896b3400/href</a>and subsequently this…
<a href="https://medium.com/media/27da95c22f9317066d392fab9f126be6/href">https://medium.com/media/27da95c22f9317066d392fab9f126be6/href</a><h4>Sequence extraction:</h4>
Contrary to what you may have thought about extracting sequences, we do not need to define another fancy UDF :) Multiple CASE statements with a simple Regex do this job pretty well. As mentioned before, they just need to be properly adjusted to catch every way the sequence is expressed.
<a href="https://medium.com/media/92785253671fe2b8331babf3c438d74a/href">https://medium.com/media/92785253671fe2b8331babf3c438d74a/href</a><h4>Levenshtein distance in practise:</h4>
As we already have the Levenshtein distance method, it is now time to use it in practice. To be able to compare one text string to another, we need to double the data by joining a table to itself using CROSS JOIN.
This operation can be very computationally heavy and take ages to run, hence the query optimisation is key, and that is why we did not want to count the precise value of Levenshtein distance once it crosses our max_boundary.
Although by using INNER JOIN, we could have the result-set produced quicker, we would need to specify what we should join the tables on. The only options would be either on director or writer. However, there are lots of sequels where both of them are different. Joining tables based on them may not fully represent our result-set, as we would probably just obtain its fraction. We would also not join the tables based on movie ids, as we would not want to compare movies to themselves.
As was said before, there is no one-size-fits-all value for Levenshtein distance, and it should always be determined with regard to the data and the problem we are working on. For this specific case, I found allowing up to 3 characters to be different a reasonable number but I highly recommend experimenting with this value on your own, if only to see how the final output can be dramatically different!
<a href="https://medium.com/media/4995b0dda8040d2481fd85456c190dc2/href">https://medium.com/media/4995b0dda8040d2481fd85456c190dc2/href</a><h4>Determining sequentiality:</h4>
To clearly indicate whether the movie is a subsequent sequel, it would be nice to add a column and express sequentiality in the form of a simple Boolean statement; TRUE — this movie is the continuation of the previous one, FALSE — no it’s not. To do so, we will just need to form specific case statements, of how the sequentiality should be determined based on seq_1, seq_2, and next_by_date.
We know for sure, in the case where seq_2 is greater by one than seq_1, we want to mark this movie as the next part. Not having first parts explicitly marked as 1, slightly complicates this task and forces us to form a slightly more complex WHEN statement: WHEN seq_1 = 0 THEN seq_1 + 1 = seq_2.
Although this should be enough in most cases, a subset of such movies may exist where the titles are near identical, the movies have been produced by the same director, and contain a sequence in the title. This is a dangerous case, where a totally different movie may be declared as a continuation of another one.
To prevent that, we are going to use next_in_sequence parameter which tells us what the subsequent movie produced by the same director was with respect to the date. To make things clear — for sequence determination, we cannot only rely on this parameter. There may have been a director who produced a couple of other movies in the gap between the sequels of the given one. This parameter, however, works well as an additional layer of certainty where the next sequel is the correct one. After incorporating it, our statement looks like this:
<pre>WHEN seq_1 = 0 AND next_by_date = 1 THEN next_by_date + 1 = seq_2 OR seq_1 + 1 = seq_2</pre>
Finally, if neither of these two statements above returns TRUE, we want to classify that movie as non-sequential. Expression seq_1 = next_by_date will do the job.
In the end our code should look something like this:
<a href="https://medium.com/media/e2bf99b42652d749978a39c4e2f4eda4/href">https://medium.com/media/e2bf99b42652d749978a39c4e2f4eda4/href</a><h4>Final result:</h4>
Once we finally have all the bits done, we can combine the puzzles using WITH AS clause and produce the table. In the final SELECT statement, we are going to select all columns from the last subquery, determine the Levenshtein distance value and eliminate those rows that have empty strings in their titles (arising out of our previous REPLACE statements). For readability, we can also ORDER the records BY title_1.
Our table of sequential movies should look something like this and has well-over 1000 records:
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mK4_6I1XRn-ixFcczNxJlQ.png"><figcaption>Figure 6 — Final table</figcaption></figure>To find out more about Datasparq, <a href="http://info@datasparq.ai/">please get in touch</a> or go to our <a href="https://datasparq.ai/contact">website</a>. We would be delighted to hear from you!!
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=4b6653b17be6" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/levenshtein-distance-as-a-remedy-for-sequential-data-4b6653b17be6">Levenshtein distance as a remedy for sequential data</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Levenshtein distance as a remedy for sequential data

<blockquote>Search for something that works and then remember it. Search and memory is the essence of Reinforcement Learning — Andrew Barto
</blockquote>
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*O45BsOLAKxpp8kfi.jpg"><figcaption>Found it! Inflatable decoy tanks were deployed on the south coast of England during WWII to fool the enemy into believing that the real D-Day landings were taking place around Calais</figcaption></figure>As a Data Science team, we were lucky enough to get into Reinforcement Learning [1] or RL a couple of years ago when we started looking at optimal pricing for car parking spaces at an international airport [2].
Airports and Dynamic Pricing
It’s an interesting problem, you have a fixed resource, the car park spaces themselves, and you have customer demand which varies over time — largely with demand for airline travel. You also give people the ability to book spaces well ahead of time. What price do you set?
<figure><img alt="An airport car park with aeroplane flying overhead" src="https://cdn-images-1.medium.com/max/1024/0*gxFrGul2iXAqrieH.jpg"><figcaption>Airport car parks presents a challenge for optimal pricing</figcaption></figure>Imagine you run such a car park and have the power to set prices. If you wait a couple of days, you might get a higher price from another customer for the same date. If you lower your price for that date because you haven’t sold the spaces and the date is approaching, your customers might quickly learn that waiting is the best policy and not book when they first check the car park website.
Furthermore, customers that book for a weekend might well be families or tourists looking for a quick break who are perhaps booking with a low cost airline. If they’ve paid £67 for a return trip to Majorca, they’re likely to think that £75 for a weekend parking spot is a little on the high side. Whereas your business traveller might think that is entirely reasonable or just not an expense they’ll be picking up at the end of the day.
The ability to learn what price works for a given demand, for a given day of the year, with a given booking lead time is a good example of dynamic pricing.
Dynamic pricing is a growing area of data science and a long-standing problem in operations research and one that is enjoying much more attention as marketplaces themselves become more dynamic.
What makes these problems amenable to reinforcement learning is vast event data sets around customer interactions. This gives an RL tool the ability to deduce the demand level (as well as, potentially, need and urgency) for a particular cohort of customers and thus find a customer specific price that will lead to a well used resource.
Why dynamic pricing?
Before we get into the reinforcement learning and how it can power dynamic pricing, we need to understand a few use-cases for why prices might need to change in more general settings.
<ul>
<li>
Increased demand/reduced supply. As in any economic model, if supply does not match demand for a product then the market may be able to sustain a higher price.</li>
<li>
Expiring products. Anything that has a limited sale window is likely to attract varying a price, whether it’s a hotel room for tonight, a loaf of bread that’s about to reach its expiry date, or a plane ticket on a flight. None of these can be sold after the window has passed.</li>
<li>
Seasonality. Some products are much more desired at certain times of day or times of year. Some may not be available at all at the wrong time. Christmas decorations typically get discounted heavily after Christmas as retailers don’t want to stock them for the whole subsequent year waiting for them to sell.</li>
<li>
Cost of production/supply. Some products cost more to produce on demand than on the normal production run. For example Brompton, the folding bike manufacturer based in London, typically has a 5 week lead time for a normal folding bike but that can be reduced to 5 days with tracked delivery if you are prepared to pay a premium for a model. The implication is that there is a reason to charge a varying price more closely linked to the cost of production (and possibly the customer’s willingness to pay for a quick delivery — see below).</li>
<li>
Price by customer. Possibly the most contentious reason to charge a dynamic price — certainly in mass retail — is the act of setting a different price for the same product to different customers. Sometimes this is veneered over by selling in different markets, e.g. electric cars in the US versus the EU market. But within a market, it may be desirable to identify customers who are time-limited and are not looking at competitor products, and may be willing to pay more for a standard product.</li>
</ul>
Varying a price to reflect super-high demand or capture a high need customer is not always the right thing to do. For instance, airlines charging thousands of pounds for seats on the last flight that makes it home before an enforced quarantine period starts or a taxi company surging prices in the face of a civil emergency — can lead to considerable long-lasting reputational damage. Important to consider— is it the ethical thing to do?
Reinforcement Learning and what to watch out for!
We’ve been working on a dynamic pricing project for the last few months. It’s a market-place environment where customers are exposed to a set of agents offering a service and we have to work out what price to set. The marketplace takes a cut of the fee paid and the service providers get the rest.
Basic features:
<ol>
<li>Customer demand is seasonal and varies dramatically across the day — it is also price sensitive.</li>
<li>The service providers are a limiting resource — if they’re working for a customer they can’t be doing other jobs for other customers.</li>
<li>The higher the price, the more the service providers want to offer the service and the quicker a match is made.</li>
<li>The lower the price, the better for the customer — of course — but also the longer they may be kept waiting for a match with a possible service provider and indeed they may get no match at all.</li>
</ol>
How do you set prices in such a setting so that you offer a fair price to the service providers and yet don’t fleece the customers. Well you can use reinforcement learning. We don’t know the price that is appropriate but we can set one and see what impact it has on the market — then we can learn and in the next epoch adjust accordingly.
<blockquote>Reinforcement learning at its root, is just memoized context-sensitive search. You do some operation and then you remember the result and then the next time you have to do that, you look it up — Rich Sutton
</blockquote>
If you read any reinforcement learning text book, you’ll find a diagram very much like this describing the basic principles. In this instance, the agent is the marketplace, the action is the ability to set a price and offer it to the customer, the state is the state of the marketplace (I know that’s self referential but we’ll revisit that) and the reward is a measure of success from having made a successful match between customer and service provider. Again we’ll come back to this because it’s not straightforward and it will drive the whole optimisation of the system. There can be a choice between many possible actions reflecting the possible prices that may be offered in the marketplace. Each action may lead to a distinct change of state and a different reward associated with that change.
<figure><img alt="A feedback loop depicting an agent taking an action, and reacting to the reward and change of state from the environment that consumed the action." src="https://cdn-images-1.medium.com/max/984/1*nQBVz20wyXh1eezFG6cVMQ.png"><figcaption>The reinforcement learning loop. CC by-SA 4.0 Jeremy Bradley</figcaption></figure>So what does the Agent do — well the marketplace learns from successful matches and unsuccessful matches and adjusts the price (or not) accordingly as each new opportunity to make a match comes round. There are maybe 20,000 transactions a day and prices are set in near real time.
I think the three most important aspects to doing this kind of activity with reinforcement learning can be summarised with:
What’s in a state?
Quick primer: Reinforcement learning comes out of a branch of mathematics called dynamic programming which solves an underlying Markov Decision Process for a value function. Deriving a value function gives you a policy which in turn tells you which actions you should take in a given state.
That was lots of jargon so let’s break that down— a value function computes the value of each state of your system under a given action policy; knowing which states are high value is important when discovering a better policy. A policy is a pre-defined strategy for choosing which action to take from a given state, ideally to maximise your accumulated lifetime reward. A Markov Decision Process or MDP is a state transition system which defines the states, actions, rewards and next states for any system.
<figure><img alt="A Markov Decision Process probabilistic transition system depicting states in green and actions in brown. Two of the actions result in distinct rewards of -1 and +5." src="https://cdn-images-1.medium.com/max/1024/0*umbiYLQFjQxM0tQ6.png"><figcaption>A simple 3-state MDP with actions in brown and rewards +5 and -1 marked. CC by-SA 4.0 Waldo Alvarez</figcaption></figure>What’s this got to do with states? If the underlying mathematical model (MDP) has too many states, then the dynamic programming algorithm will take forever to converge (or a very long time anyway). i.e. you’ll never get a solution and you won’t know what action to take — or price to set in this case — to improve your overall lifetime reward.
It is very easy to inadvertently give yourself an enormous state space. We have a marketplace that spans 850 locations across the world and we want to set prices every 15 minutes, we have 20 levels of customer demand and 20 levels of service provider supply, and 20 possible price bands. That’s a little over 4.5 billion states right there!
Surprisingly this is not necessarily a disaster as you can represent your state space approximately using a deep learning neural network of some other ML model but it definitely adds to the complexity as we’ll see when we try to learn with it.
What does success look like?
Success is defined by your reward. You might think that success is easy to specify. Why not choose revenue or profit and watch the dollars roll in. Well yes and no. Let’s imagine we used revenue in our marketplace.
If I have some very rich customers, then I can set my prices very high and I may initially even make a lot of revenue. However, I have inadvertently made my market very brittle and subject to the whims of a few well-off customers who are now dominating the market. Every other customer is priced out and will probably never come back.
So let’s lower the prices so that I open myself up to more of my customer base. The prices are still high. The revenue is improved because the customer base is larger, but there are service providers who are now turning down small jobs that are too low in value in the hope that a higher value job will come along shortly. Many customers with smaller jobs are left waiting for an unacceptable period of time before they get a price match. But that is ignored by the reward metric as it only cares about revenue in this case.
Again this is a short term win. Those customers left hanging around probably never return to your marketplace even when they have larger jobs to offer. In the end, low customer satisfaction kills your business.
What we found is that a hybrid reward metric which captures both financial success and customer satisfaction will allow you to find a sweet-spot in your problem — keep the service providers happy with a decent revenue while keeping the customers happy with a good performance metric.
Can it learn?
Reinforcement learning works best when it learns as you might expect. Successful learning comes from convergence of the algorithm and gives you a consistent result for the value of being in a particular state.
If you’re using an explicit state space model (sometimes called tabular learning, so not a neural network) you need enough training data or learning episodes to visit each state many thousands of times to get converged learning. (Converged learning refers to the model having a consistent picture of what value the most important states and actions have).
Now we know why keeping track of the state space is so crucial. If you’re using tabular learning which is simple and has many benefits you have to have a pre-training set that is 100x or 1000x the number of states in the state space or at least the number of states you are likely to visit. Alternatively, if your model is learning from scratch, your learning feedback loop needs to visit all of those states 100s or 1000s of times to get converged learning.
If you’re using an approximate state deep learning model for very large state spaces, you won’t be surprised to learn that many gigabytes of training data are needed to train that model effectively.
Certainly, if your training data only visits each state once or twice, your learning is going to be very poor and your recommended prices or actions are not going to be great!
Wrapping up
I’ve really only touched upon reinforcement learning as a mechanism for learning optimal prices. There’s lots of benefit to getting it right — an algorithm that can learn autonomously and present the right price for even new unseen scenarios; an algorithm than can even look after customer satisfaction if the correct reward structures are put in place to guide learning.
There are many ways you can get yourself in a tangle. But keeping the state space at a reasonable size that can match the available training data you have (or are likely to encounter during learning) is a great start and defining an appropriate reward metric will help you drive your learning in the right direction while not making embarrassing decisions along the way.
This introduction is still only the tip of the iceberg but it reflects some of the key decisions we had to take when taking reinforcement learning from the text book [1] (which is quite excellent by the way!) to a real-life project. Other significant considerations include:
<ul>
<li>
exploration versus exploitation — at some point you’ll need to take a suboptimal decision to see whether it leads to a better policy that was previously unknown. This can trigger some interesting conversations with stakeholders!</li>
<li>
discount factors — how much do you value reward that can’t be realised for weeks or months versus reward you can access today?</li>
<li>
learning rates — how quickly should your learning system adapt to new evidence or is your data quite noisy, thus meaning you don’t want it to adapt to a blip by mistake.</li>
<li>
algorithm — where to start; tabular or approximate representation, Expected-SARSA or Q-learning. There are many techniques and pros and cons to each of these approaches but in general, start simple and build up complexity as you need it is good advice.</li>
<li>
off-policy versus on-policy learning — is your system able to learn as it discovers new states or is it going to have to be pre-trained off-line? Is your system able to implement the decisions it takes or is it only recommending the decisions to an operator who can decide to adopt or decline them? All these factors influence whether you need to use an on- or off-policy approach.</li>
<li>
time-varying rewards and demand — probably the hardest issue of all in reinforcement learning is time-heterogeneity. Your environment may change over time and the reward you see one day may differ the next day for the same action. If this means a change in the statistical properties of your system (rather than just noise or differences in samples from the same distribution), then your RL system needs to know to adapt to that. Censored memory and rapid learning rates are the order of the day here.</li>
</ul>
All of these features need addressing at one point or other in practice. <a href="mailto:jeremy.bradley@datasparq.ai">Leave me a comment</a> if you’d like to see a Part II on some of these more technical aspects of practical implementation of reinforcement learning.
And finally…
If you are interested in joining us at <a href="http://www.datasparq.ai/">Datasparq</a> for exciting projects like this or just finding out how your company can benefit from our Reinforcement Learning, Data Science and Engineering expertise, <a href="mailto:%20info@datasparq.ai">please get in touch</a>. We’ll be delighted to hear from you.
References
[1] Richard S. Sutton and Andrew G. Barto. <a href="https://www.andrew.cmu.edu/course/10-703/textbook/BartoSutton.pdf">Reinforcement Learning, An Introduction</a>. 2nd ed. MIT Press, 2018.
[2] Andreas Papayiannis, Paul V. Johnson, Dmitry Yumashev, Peter Duck. <a href="https://doi.org/10.1093/imaman/dpy015">Revenue management of airport car parks in continuous time</a>. IMA Journal of Management Mathematics. Jan 2019.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=5f7fa3054446" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/war-stories-from-using-reinforcement-learning-for-dynamic-pricing-5f7fa3054446">War stories from using Reinforcement Learning for Dynamic Pricing</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

War stories from using Reinforcement Learning for Dynamic Pricing

Imagine you’re building thousands of rocket engines and spending billions of dollars on a quest to make humanity a multi-planetary species. If your manufacturing process isn’t as efficient as possible, you’ll run out of money before you get there. You probably need to optimise this process like crazy right? So why don’t we see if we can optimise our software in the same way…
(please ignore the fact that nobody needs to manufacture software, that’s why it’s called software not hardware, just bear with me)
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qXCFgcRXosxgEthX"><figcaption>Photo by <a href="https://unsplash.com/@spacex?utm_source=medium&amp;utm_medium=referral">SpaceX</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure>In a recent interview, Elon Musk talked about how manufacturing was underrated and was the most difficult problem SpaceX had to solve.
<blockquote>“10,000% more work goes into the production system than the thing itself.”</blockquote>
<blockquote>“The amount of effort that goes into the design rounds down to zero relative to the amount of effort that goes into the manufacturing system.”</blockquote>
He went on to describe 5 steps used to optimise engineering for manufacturing, which emerged from the many successes and mistakes of rocket programs and Teslas in the past:
<h4>1. Make the requirements less dumb</h4>
The requirements are definitely dumb; it does not matter who gave them to you. Therefore, requirements get challenged, which allows for out-of-the-box thinking (see Cybertruck).
<blockquote>“Everyone’s wrong. No matter who you are, everyone is wrong some of the time.”</blockquote>
<blockquote>“All designs are wrong, it’s just a matter of how wrong.”</blockquote>
He also states that specific people should own requirements, not departments, so that they can be questioned much quicker.
<h4>2. Try very hard to delete the part or process</h4>
You can’t afford to add a part ‘just in case’. Musk encourages deleting as many parts as much as possible, even if you end up adding some back later.
<blockquote>“If you’re not occasionally adding things back in, you’re not deleting enough.”</blockquote>
<h4>3. Simplify and optimise the design</h4>
This step comes third, not first, for good reason. The critical point of this whole process is to not do this step until you’re absolutely sure it’s needed before you start wasting your time.
<blockquote>“The most common error of a smart engineer is to optimize something that should not exist.”</blockquote>
Elon stressed the importance of an engineer understanding the whole system to avoid optimising the wrong thing. He revealed that SpaceX engineers had made the mistake of putting enormous amounts of effort into reducing the weight of engine components, but not the weight of unused fuel, despite them being equivalent.
<h4>4. Accelerate cycle time</h4>
Here’s where you start to find issues, and have a chance to fix them before it’s too late. It’s like learning to run before you run a marathon.
<blockquote>“You’re moving too slowly, go faster! But don’t go faster until you’ve worked on the other three things first.”</blockquote>
<h4>5. Automate</h4>
Why is this separate from the fourth step? Because this assumes you wait until any and all issues have been found before removing all humans from the process.
It would be hugely costly to try to automate something that was overly complicated or wasn’t needed.
This is also the point where you should aim to remove in-process testing for parts that can be produced reliably. Musk states that a tester could be creating a choke point, and will always have some have false negatives.
<blockquote>“If things are getting to end-of-line testing, and are passing, then you don’t need to do in-process testing.”</blockquote>
We can summarise this process as the following:
Design → Question → Simplify → Optimise → Accelerate → Automate
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jmz58whKh3ky9qS3"><figcaption>Photo by <a href="https://unsplash.com/@spacex?utm_source=medium&amp;utm_medium=referral">SpaceX</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>How does this relate to software?</h3>
So let’s say instead of building thousands of rockets, you’re developing the next big app/website/API. It’s a huge task. If you do everything right you’ll be bought by Google within 5 years, else you’ll take 15 years, never get feature-complete, spend all your money, and get overtaken by somebody else. How does this situation compare to SpaceX’s?
In terms of the processes and ways of thinking, there are a lot of parallels: it’s easy to get carried away solving the engineering problems and ignoring the bigger picture. Engineers tend to want to add loads of functionality that isn’t essential. They like optimise for speed and efficiency without being required to. They stick to a set of best practices and don’t usually question them.
<h4>Manufacturing ≈ Going Live</h4>
In the design stage your code only needs to work once. When you go live you’re running the same code billions of times, on millions of different computers, for years or decades. There’s a lot that could go badly: will you need constant patches, or tiny changes for every new use case? Will you need to spend forever teaching people how to use it? Will you be able to keep up with demand and scale? Only by solving these issues can you hope to actually achieve anything with your software.
Like with rockets, this is an underrated part of programming. An inexperienced software developer might assume their cobbled together script can be used in production with no changes. This is like building a rocket engine in your garage from spare parts and deciding you’re going to build 100,000 more in the exact same way.
<h4>Part = Script, Service, or Program</h4>
Cutting out an entire service from your app (e.g. deleting an email server because you can just use an API) is undeniably a great way to reduce complexity and go live faster.
<h4>
Cost = Cost?</h4>
For software, material cost is much lower, personnel cost is much higher, and there are additional computing costs. People often worry about their compute/network/storage bill, but the salaries of the people fixing bugs all day long are likely to be larger.
There’s also the opportunity cost of complexity. If a low priority feature requires a day in total to develop, deploy, debug, document, etc. over the course of a month, then you’ve missed the opportunity to start a killer feature, or present your product at a conference, or have a much needed office party!
<h4>Weight = Complexity</h4>
For rockets this doesn’t just mean weight of the part, it’s also how much thrust you can get for that weight. Adding complexity to a piece of software means more complex documentation, technical debt, bugs, deployment, etc., all of which increase costs but more importantly, slow you down and kill your momentum.
Reducing complexity allows you to go further with the same amount of resource.
<h4>Requirement = Requirement</h4>
Requirements often don’t get questioned, and they rarely have an individual owner. It’s far too easy to make assumptions about how software will be used and not seek out the opinion of an actual potential customer.
<h4>Feature = Feature</h4>
We’re all familiar with dubious feature requests. The Agile process helps to prevent low priority features from being worked on but doesn’t stop us from adding unnecessary complexity.
<h4>In-process testing = Unit tests</h4>
This one has really made me think. Elon’s right, we don’t need most of these unit tests. We’ve got hundreds that pass every time, and they sometimes get false negatives which take time to investigate. We could probably replace 90% of them with a single integration test.
(I won’t suggest getting rid of them though, just running them less often)
<h4>Colony on Mars = __________</h4>
It helps to know what you’re working towards. A clear and ambitious goal has certainly helped keep SpaceX focused on what’s important. What’s the ultimate end goal of your project? What will make you say ‘Mission accomplished’?
The optimisation of the productionisation process is vital for any software company. There’s a huge impact on time but it’s also the difference between being proud of your accomplishments, and being embarrassed. We’ve all seen projects that are being carried by one developer who knows how to fix the many reoccurring issues, and has given up hope of ever working on something interesting again.
I guarantee no app ever went live without a single issue, so there’s plenty of room to improve.
<h3>What Should We Be Doing Differently?</h3>
With all the parallels described above, I think we can take some learnings from the five steps. Looking back at some of my own projects that were difficult to get live, I can now see that we skipped some steps or did them in the wrong order, just as Elon admitted Tesla had done in the past. Based on this, here’s what I think we should all be doing differently:
<ul>
<li>Challenge all requirements, no matter how small. A common example would be “We’ve got streaming data therefore we need a streaming data pipeline” but in reality there’s no requirement for data to be processed in realtime. Micro-batch processing is much simpler and cheaper to build.</li>
<li>Pause once you have a working concept to ask which processes aren’t needed, and just delete them. Don’t spend time on a feature nobody cares about just because you want everything ‘production ready’.</li>
<li>Recognise the true cost of complexity. Any ‘part’ in your app that could be replaced with something much simpler must also be deleted.</li>
<li>Don’t optimise for the wrong thing. Is speed your priority or is it lowering cost? Should you spend your time improving features or adding more? Find a real person in the business who can explain what the priority is and why.</li>
<li>
Don’t optimise too soon. Don’t invent a framework before trying something the simple way first. Don’t define a data model when the data isn’t finalised. Be flexible to change throughout.</li>
<li>
Accelerate your code before you automate. You’ll find which parts require further optimisation, and uncover problems that would be impossible to fix after going live.</li>
</ul>
Most importantly, appreciate just how challenging it is to go from a great concept to a live app for a mass audience. Your approach to productionisation could determine your success or failure. Keep complexity low and momentum high, else you’ll never get to Mars.
The full text of the interview referenced can be found here: <a href="https://everydayastronaut.com/starbase-tour-and-interview-with-elon-musk/">https://everydayastronaut.com/starbase-tour-and-interview-with-elon-musk/</a>
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=a7963fef9887" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/elon-musks-5-steps-of-engineering-applied-to-software-a7963fef9887">What Can a Software Engineer Learn From a Rocket Engineer? Quite A Lot Actually.</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

What Can a Software Engineer Learn From a Rocket Engineer? Quite A Lot Actually.

A large part of our business is making valuable data science products a reality. I’m going to share some of what we have learnt on how to do this quickly whilst maintaining the highest quality.
Pandas alone won’t save you
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7mU7TnQZRcHy51xc"><figcaption>Photo by <a href="https://unsplash.com/@wuyyyying?utm_source=medium&amp;utm_medium=referral">Ying Wu</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure>The first mistake is to believe that every data science project is only a data science problem. Multiple different roles have an important part to play.
The best and most efficient teams I have worked in are a combination of Data Science + Product + Engineering roles. That will not come as a surprise, but what makes a difference is ensuring that these roles have a part to play throughout the project. Unless the project is heavily standardised or a repeat of another — there is virtually no chance of finishing on time or to an acceptable standard by passing (or more commonly “throwing”) from one team to another.
A common example of where early team integration really shines is during feature generation.
<ul>
<li>Data features will often need to be re-run, re-configured and re-designed (multiple times)</li>
<li>New data sources (with valuable additional signal) need to be ingested and cleaned</li>
<li>Tests on various potential models will be repeated and reviewed when features change</li>
</ul>
A role diverse team will create repeatable processes from the outset and adhere to the boundaries of company infrastructure, ensuring that no costly logic “re-writes” are required when moving from test to live datasets. The benefits are often greater than the sum of parts — we develop data models supporting current project requirements and perhaps those of others too. This is just one of the areas where we have managed to cut weeks, if not months, out of projects without effecting quality.
MLOps products help, collaboration is better
Secondly, do not let anyone tell you that an MLOps product (Kubeflow, MLflow, Cloud provided, etc)+ Data Science are going to be all the tools your team will ever need.
Elements of Data Science projects are similar, especially in operations (monitoring, testing, processing, development environments etc). Although there are great open source and cloud based tools available, parts of your projects will be unique to your business.
If your business is software based this might be second nature. If not, having a strategy to share both code and documentation between projects is a fantastic accelerator. Make sure you bring relevant teams together to discuss lightweight governance, location and features of existing reusable parts.
Here is an example of our serverless components, we use these across several projects to reduce maintenance time and improve quality: <a href="https://datasparq.ai/elements-of-engineering">https://datasparq.ai/elements-of-engineering</a>
Think about the end at the beginning
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4PFHtzQVS7nvozbt"><figcaption>Photo by <a href="https://unsplash.com/@sigmund?utm_source=medium&amp;utm_medium=referral">Sigmund</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure>Finally, those with experience will know that finishing a data science project is as difficult as starting or building one. It turns out that APIs are not designed for human consumption. And, no team is likely to be happy manually running data processes, creating graphs or having similar conversations forever.
It is also possible that no two Data Science projects you run have the same results interface. A handful of examples we have completed include: APIs, web / mobile applications, static files, databases (all different kinds), messaging service notifications and more.
Finishing a project means delivering something that can be used in operations, sustainably. It’s well worth taking the time early on in a project to make sure your team either has or can acquire the skills needed to finish the project as they might be a little different to starting and developing. This is where product-oriented awareness and design helps round a project off in a way that drives impact for the customer.
Final thoughts
Do not let any of the above put you off! Data Science projects are extremely rewarding, fantastic investments and really enjoyable to work on. The extra effort invested in early team integration, code collaboration and building tricky results interfaces is often rewarded by increased adoption and usage.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=1b380ff13ea5" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/data-science-projects-at-pace-1b380ff13ea5">Data Science projects at pace</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Data Science projects at pace

As technology and investment market trends converge, now is the time for PE firms to learn what AI really is.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mwnMr4VIcm2l4Dxka4ohhA.jpeg"><figcaption>Photo by <a href="https://unsplash.com/@yukato?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Yu Kato</a> on <a href="https://unsplash.com/s/photos/lens?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure>One clear trend in the investment world right now is the <a href="https://www.ft.com/content/cd9571a3-726c-4995-9954-23a8dcf12b19">growth of Private Equity deals</a>. The first half of 2021 has seen PE deals worth more than $500bn; helping to propel global mergers and acquisitions activity to an all-time high.
<a href="https://www.idc.com/getdoc.jsp?containerId=prUS47482321">The other trend is AI technology growth</a> — despite being around for decades, and growing significantly since the arrival of cloud computing earlier this millennium— AI is still growing. This trend isn’t particularly new, but it means that AI capabilities are becoming more common place in more businesses; and leading businesses are taking advantage of the benefits of AI to outperform their lagging competition.
Combining these trends, it’s no surprise that there’s an increasing amount of investment activity involving companies that have AI as a core component of their proposition and enterprise value. <a href="https://www.cbinsights.com/research/report/enterprise-ai-trends-2021/">AI companies raised a record $33 billion in equity funding in 2020</a>, and what’s more, an increasing number of companies claim to be “AI-first”; using cloud technology, data science and engineering to deploy AI into different aspects of their business to operate better.
Time for PE to learn about AI
If AI is deemed to be a crucial differentiator for a company’s proposition, or a major value lever given the rich, unique dataset the company has, then it’s important this is well understood when assessing the company’s valuation and investment potential. When considering a deal with an “AI-first” company, any investor needs to be clear on the existing and potential value of AI within that company.
<ul>
<li>Do buyers understand exactly what AI capabilities they’re buying?</li>
<li>How do buyers evaluate and leverage the claimed AI capabilities?</li>
<li>For data-rich companies that aren’t even claiming to use AI, how do buyers assess the potential value that AI could add to the acquisition?</li>
</ul>
Many investment brochures will advertise the organisation’s good data governance, rich data assets, scalable data platform, and leading AI capabilities that makes it a unique investment opportunity — but in many cases the truth may be it’s not so unique. After all, despite AI maturing in the last few years, it’s still like teenage sex — more companies are talking about it than actually doing it.
PE companies will use their preferred IT due-diligence and Commercial due-diligence partners to confirm a potential investment’s true capabilities, potential and risk; but these organisations rarely have the AI practitioner experience to robustly assess AI; and the remit of a typical IT DD and CDD exercise is too broad to provide any deep analysis within the AI domain.
The answer: AI Due-diligence
Given the increasing importance of rigorously evaluating a company’s AI capabilities before investing, and the lack of detail IT DD and CDD studies provide for this, DataSparQ is finding itself developing valuable AI Due-diligence reports on potential assets for PE investors. Whilst a PPT slide deck isn’t the traditional output of a DataSparQ project (we pride ourselves on developing and running operational AI solutions in the real world; our outputs are usually more Python than PPT), our deep AI practitioner expertise means we know what good looks like, and because we understand what it takes to deliver AI successfully in the real world, we’re in a privileged position to help others.
Our AI DD service consists of both a “backward-looking” confirmatory assessment of a potential investment’s AI capability, as well as a “forward-looking” analysis of how AI could be used to unlock further hidden value in that business. Different investments often have a different focus; for businesses with established AI capabilities that give them a unique competitive advantage, the confirmatory assessment is more important. For data-rich companies early in their analytics journey, the forward-looking analysis is often more valuable.
The areas of investigation during an AI DD include:
<ul>
<li>
Data diagnostic — a review of the data collected, stored and used by the business. Is it done systematically or ad-hoc? How accurate, relevant and complete is it? How is it catalogued and governed?</li>
<li>
Algorithm diagnostic — an assessment of the algorithms and analysis used. Are they appropriate (meeting requirements for accuracy, explainability, auditability, bias, customisation)? Are they robust? Will they scale and remain competitive as the business grows and the market matures? Are they compliant with industry rules and regulations?</li>
<li>
Technology diagnostic — a review of the technology systems and architecture used to build and run these AI capabilities. Are they resilient? Will they scale to support future growth requirements? Are they cost efficient? Are they secure?</li>
<li>
Capability diagnostic — an analysis of the business’s way of designing, building and running AI solutions. Do the right skill-sets exist? Is the capability unique and sustainable? Does the way of working encourage innovation?</li>
<li>
AI Opportunity review — a process to identify, validate and quantify high-value opportunities for AI yet to be exploited. What hidden value levers can AI unlock? What incremental value does AI offer over a more low-tech solution? How feasible is each opportunity? What’s a realistic delivery timetable for value capture?</li>
</ul>
Given a modest amount of time with the management team, and other key individuals in the business who might not normally be involved in traditional DD exercises, these questions can be answered within the timelines for most DD processes. It can be efficient for the AI DD provider to work alongside the IT DD and CDD providers so the same questions don’t have to be asked twice.
Bottom line
As AI matures and becomes a differentiating value lever in organisations, it’s increasingly important for investors to put this capability under the microscope when considering deals. This means PE companies need to start asking new questions, and learning more about AI than the buzzwords and headlines in order to know what questions to ask, let alone answer them.
The innovation of AI technologies and methods is fast and shows no sign of slowing. It will be an ongoing learning journey for investors to ensure their assessments remain relevant. As a result, AI practitioners like us may find ourselves increasingly doing PPT as much as Python as we become the go-to service providers for AI DD.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=e2da04f94b8e" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/why-private-equity-needs-to-learn-about-ai-to-make-better-investment-decisions-e2da04f94b8e">Why Private Equity needs to learn about AI to make better investment decisions</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Why Private Equity needs to learn about AI to make better investment decisions

Humans don’t like risk and, as with any technology investment, a Machine Learning solution comes with its own set of risks. Will it predict the right thing, will it be accurate, will it be useable? In this post I’m going to take you through how we at Datasparq reduce risk through Proof of Value.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*IwpfqunBgcA0693VDTVP4g.jpeg"><figcaption>A machine learning solution to demand forecasting can lead to pretty, ordered, stocked shelves… and higher revenues! But will it work?</figcaption></figure>Our machine learning solutions are designed to allow people to perform beyond their potential, whether that’s making more accurate sales forecasts, creating effective retention strategies for customers, or focusing business development effort on leads likely to convert to wins. Machine learning solutions give people the benefit of the predictive power hidden away in data.
Machine learning has proven time and again to be effective at predicting the future by learning from the past, whether that’s <a href="https://www.bbc.co.uk/news/technology-57157566">identifying potential skin cancer</a>, or <a href="https://towardsdatascience.com/deep-dive-into-netflixs-recommender-system-341806ae3b48">helping you decide what to watch next on TV</a>. That’s not entirely surprising as these solutions are fundamentally doing what humans have evolved to do — observe, learn and predict, just with the advantage of being able to process vast amounts more data.
But just because there’s evidence something has worked before doesn’t necessarily mean it’s going to work in every situation.
This is why we start all of our machine learning projects with a Prove phase. It’s a short, targeted mission to reduce ambiguity and test assumptions to minimise risk. Over the course of a few weeks, we validate or test the following:
<ol>
<li>That there is a clear target variable for the prediction - we know what we’re predicting.</li>
<li>That there is a viable means for measuring value from the prediction</li>
<li>That there is predictive power in the data (signal in the noise)</li>
</ol>
1. Clear target variable for the prediction
The target variable is the variable in a dataset whose value is to be predicted using other variables. It’s pretty fundamental to any machine learning solution, and picking the wrong target variable can lead to significant wasted effort down the line.
Our goal here is to define which fields in the data will be used to form the ‘target variable’. This is an iterative process where we identify a target variable, then validate how a prediction of this target variable will be beneficial. This in turn raises new questions which allows us to loop back and idenfity a better target variable.
It sounds like it should be easy. Let’s look at demand forecasting as an example. Imagine you sell jeans in a high street store. In order to effectively manage your stock levels you need to make an estimate in advance of how many you’re going to sell. So a machine learning solution just needs to predict your sales. Simple, right?
In reality, defining the scope of prediction tends to be a significant chunk of work that requires us to deep dive into an organisation’s processes and value chain.
<ul>
<li>What product or service are we predicting demand for? Is it one product, or hundreds, each with different demand patterns?</li>
<li>For what period are we predicting demand — next week, month, year? Or maybe combinations of these, or maybe different products would benefit from different levels of prediction?</li>
<li>What is ‘demand’? Is it a placed order? A product taken off a shelf? What if that order was later cancelled, returned, or not paid for?</li>
<li>Do we need to care about who is making the demand — do we need to predict demand from a specific customer or location, or a demand that will be fulfilled by a specific fulfilment centre?</li>
<li>How far in advance would the prediction be required in order to be valuable to the business? Predicting tomorrow’s demand today is much easier than predicting for a day next month.</li>
</ul>
Some of these questions (the known unknowns) will have been answered prior to the Prove phase as part of our <a href="https://www.datasparq.ai/products/">SparQshop™</a>, others arise from deeper exploration of the data (the unknown unknowns!). All need to be answered if we’re to precisely define the target variable.
2. Measuring value
In order to optimise a model we need to know what metrics we’re using to measure performance. Understanding the key performance metric gets to the heart of some of the difficult questions around model optimisation, for example:
<ul>
<li>In a regression problem (where we are predicting a number, like product sales), what’s the relative value in over predicting vs under predicting? Underpredicting by 10% might have a significantly bigger cost impact than overpredicting by 10% (you can’t sell what you don’t have, you may though be able to store some excess stock for selling later).</li>
<li>In a classification problem (where we are predicting a binary outcome, like customer churn), what’s the impact of a true positive vs a false positive? If we predict a customer to a subscription service is going to leave (churn), they may get offered a discount to stay. If they take up this discounted offer that’s great if they would have otherwise left. However, if it was a false positive prediction and they were actually never going to leave, it’s a cost to the business.</li>
</ul>
Then there may be other factors specific to the problem being solved. In the demand forecasting example, is it more important to accurately predict demand for some products over others (maybe some have a shorter shelf life, or are more expensive to keep in stock)?
The goal here is to come up with a metric to measure model performance which relates directly to the value being created for the business. For example, for customer churn, that metric may be:
Profit ($) = R*TP — C*FP
Where R is the revenue from a true positive, C is the cost of a false positive, and TP &amp; FP are the true positive and false positive rates.
3. Predictive power in the data
This is where we create a baseline machine learning model and test the predictive power inherent in the data. This starts with some basic evaluation of the data — specifically is there enough historical data to be able to train and test a model?
Next we need to know where to start with the data. We’ll typically run a hypothesis workshop with people in your organisation who understand the data and the target variable. In the case of demand forecasting this would involve those close to the buying/selling operations to understand, from their experience, what factors affect demand. For example, we may learn that they see sales go up in summer, or are higher at shops in certain locations. From these hypotheses, we extract <a href="https://en.wikipedia.org/wiki/Feature_(machine_learning)">features</a> from the data, for example “store location”, or “average temperature over the previous 7 days”.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/459/1*Uny-ym1ashJxGKzn1Gmwhw.png"><figcaption>thanks <a href="https://xkcd.com/license.html">XKCD</a></figcaption></figure>From here we can do some univariate analysis. Are there features which show predictive power on their own? <a href="https://www.tylervigen.com/spurious-correlations">Correlation ≠ causation</a> so success here isn’t proof of signal in the noise, but can help steer focus to the most useful areas of the data.
Once we have a handful of features we believe to be predictive, we can put them into a baseline model and test it. We measure the performance of the model by testing it against data it hasn’t seen. This is both out-of-time (given complete data up to 2018, can it predict what happened in 2019) and out-of-sample (given data for all time but removing data for certain shops, or products or price points at random, can it predict the gaps?).
The Prove phase isn’t designed to deliver a fully optimised model, our goal here is to demonstrate evidence that there is predictive power in the data and to get a sense of how much value might be gained through an optimised model. It’s not an exact science, but working closely with the subject matter experts gives the model the best chance of succeeding early on and it’s usually possible to prove value in this first phase, or at least get a good sense of whether it’s worth pursuing.
We’re good to go!
…not quite. The above activities bring clarity and definition to the model and it’s potential value but even the most accurate demand forecast is of absolutely no value unless it’s used.
The final piece we look at in the prove phase is how the solution will be used in operations. Are the stakeholders bought in to the idea of having a machine help with their jobs? Is is feasible to integrate the predictions with the systems/tools they use every day? What changes might be needed to operational processes once the predictions are live?
Even if these questions aren’t fully answered at this stage, having the discussion early on is another way to reduce risk.
If you think there may be an opportunity for machine learning to deliver value in your organisation and you’d like to chat about a low risk way to get started and prove the value, please get in touch at <a href="mailto:info@datasparq.ai">info@datasparq.ai</a>.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=e61f9700ef58" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/proving-the-value-of-machine-learning-e61f9700ef58">How to prove the value of machine learning and avoid worthless models</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

How to prove the value of machine learning and avoid worthless models

Well you probably do, it’s not that hard.
Rather than test your knowledge of the language, I’ll suggest seven different things you could do or learn next that will make you a better, more capable, Python developer. After all, it’s not the language itself that’s powerful, but how it’s used!
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2Uw3ZfvoggULk3qj"><figcaption>I promise I’m not gatekeeping (photo by <a href="https://unsplash.com/@lailagebhard?utm_source=medium&amp;utm_medium=referral">Laila Gebhard</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a>)</figcaption></figure><h3>1. Master Unix and Learn (Some More) Bash</h3>
Hopefully you already love Bash more than Python and use it every day, but for many devs who started out with Python, it may never have been a major part of everyday operations. If Bash seems scary or clunky to you, I strongly recommend experimenting with some core commands and understanding loops, piping, and links.
The command line is the universal standard for getting programs to talk to each other, and if you can’t get your Python to talk to other programs, you’ll repeatedly hit barriers.
Bash has helped me get around awkward situations like not being able to use pip (in a serverless code execution environment), and not having admin rights on Windows (Git Bash was installed, thankfully).
You can always rely on Bash. With a strong grasp of it you can work around any problem. Not to mention you can run Bash commands with Python and vice versa.
<h3>2. Learn a tiny amount of HTML/JS/CSS</h3>
There are some things Python should not be used for, and Front End development is one of those things.
I’d also say the same about data visualisation. My favourite plotting libraries for Python (<a href="https://plotly.com/python/">Plotly</a> and <a href="https://bokeh.org/">Bokeh</a>) render charts within a browser, using JavaScript. These achieve far better results than Matplotlib in terms of appearance, ease of use, and interactivity.
JavaScript is even eating into Java and C#’s market share. <a href="https://www.electronjs.org/">Electron</a> allows you to build desktop applications with just HTML and JS (building on top of Chromium). It might surprise you to learn that MS Teams is an Electron app.
Don’t pigeonhole yourself as a purely back-end dev! Learn the basics of a framework like <a href="https://reactjs.org/docs/create-a-new-react-app.html#create-react-app">React</a> and start showing off how great your work is with some stunning user interfaces. Plus, putting some thought into the user experience of your app will help streamline your back-end code.
Best of all, you get to call yourself a Full Stack developer.
<h3>3. Use Docker to industrialise your code</h3>
<blockquote>“Well it works on my machine” -Bad Python developer</blockquote>
Hopefully this one seems obvious. A Python developer running everything on their laptop, or on a VM with a lengthy setup script, is not a good developer.
Code that provides value should be able to run on its own, without being babysat, and should be able to be deployed and redeployed by anyone, otherwise it will end up being thrown away as soon as the person who wrote it isn’t around.
The first step to industrialising code is to move the code and all dependencies into a container. Docker makes this fairly straightforward with Dockerfiles, and allows anyone to run the container on any machine, provided they have Docker installed.
You can then make use of serverless tools like Google’s Cloud Run or AWS Lambda, which will run your container on request without the need to provision compute resources. This is the easiest way to turn your code into an API, and allow anyone to use without having to install anything. From here you can plug it into a web front-end and reach a much larger audience.
I’ve seen many git repos from universities and hobbyists alike that contain some amazing code but have absolutely no thought put into industrialisation. This is a great shame since it prevents most people from being able to play with (and therefore potentially contribute to) the code. If they had only provided a requirements.txt and Dockerfile, someone might have taken it and created something truly mind-blowing.
Disclaimer: I’m a huge fan of Cloud Functions/Lambdas which don’t require any Docker knowledge, but some tasks require containers.
<h3>4. Learn PostgreSQL
</h3>
Everybody knows that Python is great for transforming data but that doesn’t mean it’s a great choice for an application backend or data pipeline. In fact, I would strongly discourage the use of Python (or pandas specifically) for any scenario in which a database could also be used. Database technologies like Postgres and MySQL have been designed to deal with data, and they do it really well.
Next time you think about joining two datasets with pandas, consider spinning up a PostgreSQL instance, importing your data into tables, and running a join on those tables (scripted by Python, using <a href="https://pypi.org/project/psycopg2/">psycopg2</a>). The setup may take longer, but when you find yourself needing to repeat the join again with updated data, you will see the benefits in terms of speed and reliability. You can then easily parameterise your SQL with Python formatting or Jinja2, massively reducing the amount of code needed to carry out all of your transformations.
SQL is dead easy to learn and Postgres is a great starting place. Once your data outgrows Postgres, you can apply the same knowledge to other, more powerful tools like Google BigQuery or Snowflake.
<h3>5. Don’t repeat yourself… ever again in your entire career</h3>
Have you ever copied code from one project to another? Perhaps you do this at the start of every project.
Save others from repeating your work! Make packages.
The first step is to collect together all the useful functions you use regularly. It might be best to group them up and make lots of separate packages to keep requirements simple.
Secondly make a <a href="https://docs.python.org/3/distutils/setupscript.html">setup.py</a> so that you can pip install your package locally.
Thirdly, and most importantly, put your package into a git repo and share it with your team*. Python packages can be pip installed directly from git repos like so:
<pre>pip install "git+ssh://git@github.com/&lt;username&gt;/&lt;repo&gt;.git"</pre>
Once you have a whole company using the same code across multiple projects, you start to see huge time savings both when starting new projects and when upgrading/fixing bugs in commonly used tools. I dread to think how long it would take us at DataSparQ to patch each of our projects separately if we weren’t using the same packages across all of them.
*Or, if you’re happy for it to be public, upload it to PyPI. That way you can save everyone from repeating your work!
<h3>6. Become a Git Expert</h3>
I’ve met developers who use Git every day and don’t actually understand it. As soon as something unexpected happens they don’t know what to do, and don’t know how to avoid problems in the first place.
Every team needs a Git expert. Someone who can confidently handle tricky merges, rebases, cherry picks, etc., and keep repos tidy.
If your releases are difficult and technical debt is piling up, the problem might be here. Make sure you understand branching strategies like Gitflow, and how to use Git + Agile + good documentation in the most complementary and efficient way.
<h3>7. Learn C to remove bottlenecks</h3>
I have come across problems that Python was too slow for. The solution was C, wrapped in Python (so you could say Python wasn’t too slow).
Ok you don’t need to learn C. I would suggest Go instead as it has a lot of support from cloud providers and is much easier to compile for different architectures, which was always the biggest pain with C for me.
Some of the most powerful packages like numpy, pillow, and TensorFlow are all just C modules wrapped in Python. That level of performance can also be achieved with very little effort these days. One could write a simple program in Go and call it from Python as part of a larger script, for example:
<pre>import subprocess</pre>
<pre>def run(cmd): p = subprocess.Popen(cmd, stdout=subprocess.PIPE, text=True) return list(iter(p.stdout.readline, "")) </pre>
<pre>x = run("count_emojis 🚀 --file wikipedia.zip")</pre>
If anything, writing a hello_world program in C is worthwhile purely to appreciate how easy Python is.
Disclaimer 1: I only suggest this method for simplicity. It doesn’t benefit from shared memory but is much less painful to write, and less likely to make Python crash.
Disclaimer 2: Python is definitely powerful enough to count emojis.
<h3>Conclusion</h3>
There are 0 job roles that require Python skills and nothing else. Branching out and learning complementary skills will make a huge difference to how effective your code can be.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=9c0d8dc8451e" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/so-you-think-you-know-python-9c0d8dc8451e">So you think you know Python?</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

So you think you know Python?

Synthetic datasets are boring. In most cases, we just generate them by drawing samples from a multivariate distribution with components from families of distributions that we have set previously, adding some noise at the end to make them more realistic. And of course, modelling a synthetic dataset is rather easy, as we have access to the underlying distribution of the data.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rlI8fkpCKdzYg7ljfQlclA.jpeg"><figcaption>Photo by <a href="https://unsplash.com/@photography_by_feaver?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">James Feaver</a> on <a href="https://unsplash.com/s/photos/real-estate-uk?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure>Real datasets are much more exciting. We are given some data and the task of modelling their underlying distribution. There are no right or wrong answers, and every approach is fair game. There are obviously some better and worse solutions which is precisely what we try to explain in this post.
Consider <a href="https://www.kaggle.com/hm-land-registry/uk-housing-prices-paid">this</a> dataset from UK’s HM Land Registry, hosted in Kaggle. It contains more than 22M real estate transactions for the period between 1995 and 2016, with individual information on the price of sale, date, location and type of property, among others. From the original dataset, we select the period between 2012-2014 and consider only the columns that describe the date of sale and price.
<h3>1. Objective and Approach</h3>
Our aim is to model the probability distribution of prices paid for real estate in the selected period. We examine four different approaches to tackle this problem:
<ul>
<li>Gamma fit</li>
<li>Log-normal fit</li>
<li>Gaussian Mixture for the log-price data</li>
<li>Gaussian Mixture</li>
</ul>
In this post, we review mixture models [1] to show how these can be useful when dealing with complex probability distributions. In particular, we apply a Gaussian mixture model to model the probability distribution of prices. The code to run these simulations can be found <a href="https://gist.github.com/kiko-datasparq/3070ad063dd3b69d240f23181c00126a">on github</a>.
<h4>Take-Home Messages</h4>
What worked? Gaussian Mixtures provided a great balance between mathematical complexity, difficulty in the implementation and results for the fit. What didn’t work? I initially tried fitting a mixture of Gammas because I thought it might give better results. This became a mathematical nightmare that I quickly dropped, sticking with the nice closed-form results for Gaussian mixtures. What was time-consuming? I spent a significant part of the time understanding and interpreting the components in each GM fit, as well as finding a nice way to demonstrate my insights. As a result, I decided on the log-log, log-x and linear plots to show each part of the distribution in detail. Why should we care about this? Modelling the underlying distribution is crucial to understanding our data. Although in many situations we might get away with a known distribution, this is not always the case, as we have clearly shown. Having mixture models in our toolbox allows us to work with more complex datasets.
<h3>2. Time-adjusted Prices</h3>
Let’s examine the evolution of the mean daily price paid per property in the selected period.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S9J8Wx4rhsC9UQesXhMA7Q.png"></figure>It is very clear that there is an increasing trend in this time series. The series starts with mean values around 230k and ends with mean values in the order of 280k, implying an increase of around 20%. Before modelling the probability distribution for the whole series, we adjust the prices for this increase.
We model the trend using an exponential function of the form
<figure><img alt="" src="https://cdn-images-1.medium.com/max/141/1*oA6BtHb9GmiRbP51JcLuIw.png"></figure>where a, b are the parameters we need to fit, t is the time (in days) since the beginning of 2012 (our selected period) and y is the target price (in £). The result of the fit is given below.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KoiNkzEgDmqieHyakpJCsg.png"></figure>An exponential fit provides a model with an intuitive interpretation in this context, where we have a geometric growth (similar to inflation) of prices. Note that a linear fit would give a very similar result but would lack such interpretability.
To adjust the prices, we multiply the price of the sale by exp(-bt), where b is the parameter obtained from the fit and t is the number of days between the beginning of 2012 and the actual day of the sale. As a result, we obtain an adjusted time series that looks as follows:
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*phu1H0daPdfKQmCeMAK_1w.png"></figure>This result is far from perfect because we still observe periods where the adjusted price goes far away from the mean (which corresponds to parameter a from our exponential fit). However, it is a significant improvement with respect to our original data because we have eliminated its trend.
<h3>3. Modelling the Price using a known Distribution</h3>
We’ll assume the exponential adjustment is sufficiently good for our analysis. Now let’s examine the distribution of adjusted prices.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ddlZe5-sIS_g6IJul-S3iQ.png"></figure>This histogram has been truncated to prices below £1.2M and grouped in bins of £1k. The price distribution is relatively smooth and has a heavy tail with a maximum above £80M (not shown in the histogram).
<h4>3.1 Gamma Fit</h4>
As a first approach, we model the probability distribution using a Gamma. Note the pdf for this distribution is given by
<figure><img alt="" src="https://cdn-images-1.medium.com/max/473/1*AHDH3IHEJv5SlARCEkuH4g.png"></figure>where α, β and x₀ are the parameters to be estimated. It turns out that for our data a Gamma distribution is not a good model. The Maximum Likelihood Estimation (MLE) with the entire distribution does not give anything meaningful so we don’t include that result here. The fits with percentiles 95 and 99 (p95 and p99) of the data are given below.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aE_6qg_B-0-VIeih2NpeZw.png"></figure>The p95 fit captures well the mode of the empirical distribution but it does not explain a significant proportion of the data. As we try to include a larger part of the tail, the model breaks. We see that for p99, the match between the empirical and Gamma fitted distributions is much worse than for p95.
<blockquote>In conclusion, the Gamma distribution is poor at capturing the heavy tail of our data.</blockquote>
<h4>3.2 Log-normal Fit</h4>
The histograms for the price (with log-x axis) are given below. For the plot on the left, we have also used a log-y axis to clearly show the heavy tail of the distribution, with prices above 1M for tens of thousands of transactions.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eNqWOcLTniISmGyqDDhvVQ.png"><figcaption>(left) Log-log histogram for price data; (right) Log-x histogram for price data.</figcaption></figure>The plot on the right resembles a Gaussian distribution which motivates using a log-normal distribution to model the prices. To this end, we transform our original price distribution X as
<figure><img alt="" src="https://cdn-images-1.medium.com/max/125/1*PSF3VRG1rbKaRjPlajEqCw.png"></figure>where
<figure><img alt="" src="https://cdn-images-1.medium.com/max/132/1*DsYavMJHktPrdUjv_9QT_g.png"></figure>is distributed as a Gaussian with mean μ and standard deviation σ. We compute the MLE fit with the log transformation of the data and obtain the following results.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k27zcg74csn2exEdiD2qdg.png"><figcaption>(left) Log-log pdf; (centre) Log-price pdf; (right) pdf truncated at 1.2M.</figcaption></figure><ul>
<li>The plot on the left shows the log-log pdf for both the empirical distribution (i.e, the data) and the log-normal fit. We observe that for selling prices larger than £1M, the fit systematically underestimates the observed probability.</li>
<li>The centre plot shows the log-x pdf. The log-normal fit captures correctly the mode at around £100k. However, we see that it underestimates the probability allocated at the mode. We also observe this in the linear-x plot on the right.</li>
</ul>
<blockquote>We are interested in finding a model that can approximate better the pdf both at the mode and the tail of the distribution. One way to do this consists of using mixture models which we review next.</blockquote>
<h3>4. Modelling the Price using Gaussian Mixtures</h3>
As we saw, a log-normal fit for our property prices data presents certain limitations, since the two free parameters that we fit in the MLE cannot capture all the complexity present in the data. Mixture models provide a solution to this problem. We assume that our data X is distributed as the linear combination:
<figure><img alt="" src="https://cdn-images-1.medium.com/max/202/1*kq0cNeP0FWJPs0JQZ_mJLQ.png"></figure>where the weights ωₖ satisfy
<figure><img alt="" src="https://cdn-images-1.medium.com/max/108/1*jz8vLVvvmKkxi76oqsAUJg.png"></figure>and the basis functions gₖ are probability density functions (pdf) from known families of distributions. In this case, we choose gₖ to be Gaussian, therefore:
<figure><img alt="" src="https://cdn-images-1.medium.com/max/161/1*2mE9TpdbVzXd4EErJQCuxQ.png"></figure>Instead of estimating (μ, σ) as we did before, now we estimate (ωₖ, μₖ, σₖ) for all components in the mixture. In the numerical experiments below, we have used the <a href="http://pages.cs.wisc.edu/~jerryzhu/cs838/EM.pdf">Expectation-Maximisation (EM)</a> algorithm to obtain the parameters for the mixture which is very simple and easy to implement.
<h4>4.1 Gaussian Mixture for Log-price</h4>
We set the number of components in the mixture K = 6 and find the MLE solution for a Gaussian Mixture (GM) for Y = log(X), where X is the adjusted real estate prices. This value has been chosen heuristically to strike a good balance between the number of parameters (model complexity) and the capacity of the model to explain the data. We briefly address how to quantitatively select the number of components in the mixture in “Open Questions”.
<h4>4.2 Gaussian Mixture for Price</h4>
We set K = 6 and find the MLE solution for a Gaussian Mixture (GM) for X, the adjusted real estate prices.
<h3>5. Results</h3>
So far we have discussed four approaches to model the probability distribution of real estate prices, namely:
<ul>
<li>Gamma fit</li>
<li>Log-normal</li>
<li>Gaussian Mixture for the log-price data</li>
<li>Gaussian Mixture</li>
</ul>
We discard the Gamma fit from further analysis due to its severe limitations and compare the remaining three. In the graph below, we have plots with four lines, one corresponding to the empirically observed distribution (data pdf) and one for each of our fitted models.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lzQacZTTQoSh9goOjc4WKQ.png"><figcaption>(left) Log-Log pdfs; (centre) Log-x pdfs; (right) Pdfs truncated at 1.2M.</figcaption></figure>As we already analysed the log-normal fit, let’s discuss the GM for the log-price (green curves). This fit is clearly better. In the (left) log-log plot we observe that the GM-log fit is closer to the empirical pdf than the log-normal fit. Additionally, in the centre and right plots, we note that the mode is not only captured but also assigned the correct probability. The only limitation of this mixture is that it systematically underestimates probabilities for prices above £5M.
The GM fit for the price gives slightly different results. It is also preferable to the log-normal fit since it follows the tail of the distribution for large prices, and it assigns the mode the correct probability, as we see in the centre and right plots. However, we also observe that at the head and the tail of the distribution, there is still a discrepancy to the empirical pdf.
<h4>5.1 Analysis of Components</h4>
In both GM fits we chose K=6 components. We now have a look at how these components look and contribute to the final GM distribution.
In the graphs below, we have plotted a line for each component gₖ in the GM log fit, weighted by its corresponding ωₖ, each denoted by Cₖ for k=1…K (not sorted in any particular order).
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9SoDjcW83SG1KBOh63NatA.png"><figcaption>[GM log-price components] (left) Log-log pdfs; (centre) Log-x pdfs; (right) Pdfs truncated at 1.2M.</figcaption></figure>These plots are quite revealing. The (left) log-log plot shows that the components present a very sharp decrease at the tail of the distribution which is why this mixture struggles to capture probabilities for large prices. The plot on the right makes explicit that each component is a log-normal distribution in the linear space (as expected).
Similarly, we analyse the components for the GM fit for the price. In the graph below, we have plotted the pdf for the weighted components ωₖgₖ, each denoted by Cₖ.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lTc2GTJui_a2scydDLfpuw.png"><figcaption>[GM price components] (left) Log-log pdfs; (centre) Log-x pdfs; (right) Pdfs truncated at 1.2M.</figcaption></figure>The plot on the right shows that the pdf is decomposed using Gaussians in the linear space. In the plot on the right, we see why the pdf has wiggles, as we have a component (C1) with a very large mean (more than £5M) with a very small weight which contributes to allocating probability at the tail of the distribution. This is something we did not have either in the log-normal or the GM-log fits, as these fits systematically underestimated the probability at the tail of the distribution.
Additionally, the GM fit assigns a non-zero probability to relatively small, non-negative values, which are unrealistic (all price values in the dataset are strictly positive, as expected). However, for consistency, we have excluded prices lower than zero from the linear-x plot on the right.
<h4>5.2 Open Questions</h4>
There are different directions for improving this analysis. A fair question would be: how do we choose between models? As of now, We do not have a clear answer on whether the GM-log or GM (or even the log-normal) fit is preferred. We discussed the results qualitatively but did not provide a metric to make a quantitative decision. Additionally, we could ask how best to choose the number of components. Both these questions can be answered via the <a href="https://medium.com/@analyttica/what-is-bayesian-information-criterion-bic-b3396a894be6">Bayesian Information Criterion (BIC)</a> which allows choosing between models with a different number of parameters via computation of the likelihood.
<h3>Summary</h3>
<ul>
<li>
Mixture Models provide a flexible framework to model complex distributions coming from real data.</li>
<li>We used Gaussian Mixtures (GM) to model the distribution of real estate prices in the UK for the period 2012–2014. We compared GMs against more simple distributions such as Gamma or Log-normal which provided worse results for the fit.</li>
<li>
Gaussian Mixtures are simple and easy to implement in combination with the EM algorithm. There are analytical expressions for estimating all parameters which is not the case for other mixtures.</li>
</ul>
<h3>References</h3>
<a href="https://www.gwern.net/docs/statistics/2019-mclachlan.pdf">[1]</a> McLachlan, Geoffrey J., Sharon X. Lee, and Suren I. Rathnayake. “Finite mixture models.” Annual review of statistics and its application 6 (2019): 355–378.
Would you like to understand how AI can generate value for your business? Visit <a href="https://www.datasparq.ai/">DataSparQ </a>to find out more about the products we could build for you and the services we offer or get in touch with a member of the team to start your AI journey today.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=d70acb3bc9d" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/gaussian-mixtures-to-model-real-estate-prices-in-the-uk-d70acb3bc9d">Gaussian Mixtures to Model Real Estate Prices in the UK</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Gaussian Mixtures to Model Real Estate Prices in the UK

A small bit of web history
Some years ago in the 90s when I was doing my MSc in Computer Science and the web was the next big tech frontier, we had a problem.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fbQmx9uT6PuwrmE3Vv36BA.jpeg"><figcaption>Photo by <a href="https://unsplash.com/@dlerman6?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Daniel Lerman</a> on <a href="https://unsplash.com/s/photos/search?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure>That problem was search — it was broken, not just a little bit either but totally broken. The search engines of the day regarded search much as a word processor does now. They gave you a search dialogue and expected to return some or even all of the pages in their catalogue which contained your search term. If there was a tie between matching pages, then the engine might look to see if your search term appeared in a title or heading or appeared multiple times within the body of the page.
The inevitable happened — spam pages emerged with huge numbers of hidden words and likely search terms embedded in them. Overnight web search as a tool became completely useless. You couldn’t find anything.
That was of course until <a href="http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf">Sergei Brin and Larry Page wrote a paper</a> as part of their PhD detailing a neat piece of what would later become known as Data Science that could recover relevance and importance in web search. That piece of applied mathematics was PageRank and it became a $1tn company — you will know it as Google.
Job search
Well searching for a job is a bit like that now. Job postings appear containing all sorts of relevant or not so relevant information. They’re posted with wildly diverging titles even if they represent essentially the same job. So Primary Teacher role might get posted with the title: “Part-time Teacher (Primary) years 5–6 Ipswich-starts September” or equally “Primary Teacher (Fixed term maternity cover) £24k — Brent”.
All because jobs forums and recruiting companies think they can elevate their posting in a simple search box. Meanwhile applicants are reduced to searching for terms they think might match the sort of job they are interested in — often the job title. The result is a deeply unsatisfying user experience —lists of over specific jobs that do not represent the breadth of possibility for the applicant’s capability.
Job search is as broken now as searching for a web page in 1996.
The Job Finder Machine
We started work on the <a href="https://labs.datasparq.ai/job-finder-machine/">Job Finder Machine</a> to address this problem in December 2020. As my colleague <a href="https://medium.com/@oliverbartlett/1a7ffe6aa7ff">Oli Bartlett has written in his post</a>, this was part of a data-science-for-good project with the <a href="https://emergentalliance.org/products-and-services/job-finder-machine/">Emergent Alliance</a> to answer the challenge of getting people with skills into well-matched jobs after the pandemic.
Our Data Science Hypothesis
Our data set consisted of a set of job titles representing current or recent job openings and associated with each title was a list of skills that were required by that job.
As with all good Data Science projects, we started with a working hypothesis which we tried to disprove.
<blockquote>
Hypothesis: We can generate high quality job matches based on user-entered natural language descriptions of skills alone</blockquote>
There were the usual problems of skills being duplicated, spelt wrong, capitalised, not linked with the skill with exactly the same name. Also we were expecting user input of skills from the applicant with exactly the same issues. Some useful techniques from NLP came to hand to help address some of these issues — but not all.
How did we do it?
First we developed a conceptual model of the jobs-skills landscape: a bipartite graph connecting a set U of jobs with a set V of jobs.
<figure><img alt="A bipartite graph representing the relationship between job postings and skills" src="https://cdn-images-1.medium.com/max/1024/0*nPps9WEKeqlJ95Rd"><figcaption>A graph of U, the set of job postings, to V, the set of skills needed by a job.</figcaption></figure>Why bipartite? Because at this stage, we only had a relationship between jobs and skills or skills and jobs and there were no direct relationships between jobs (this would come later).
What did this represent? It showed simply that some jobs need a set of skills and those skills might also be required by some other jobs. Not earth shattering but still useful.
Imagine that an applicant highlights some skills in set V that are relevant to them, could we use that information together with the graphical model to return a list of jobs that were a good match for the user’s skills. Potentially.
Those sets, by the way, U might have as many as 180,000 current jobs in at any one time, and V might have 3 million skills listed.
Just returning all the jobs that happened to mention one of the skills the candidate mentioned was a non-starter — we would have many 1000s of matching jobs potentially.
Equally, we realised quite quickly that returning all the jobs that matched all the skills a user mentioned quickly reduced the the number of matches to zero and even if not, very clearly missed lots of jobs that were clearly very similar in nature but had happen not to mention one of the skills that other jobs with the same name had mentioned.
Step 1: Constructing a common skills description for a job
<figure><img alt="Job descriptions and their associated and likely different skills descriptions." src="https://cdn-images-1.medium.com/max/1024/0*DF9kAkcbepxq7d21"></figure>Many jobs with the same job title had slightly differing skills.
We were trying to give a good match to a set of likely jobs and not necessarily a precise match to a specific posting. Also it might be that a hiring manager had left off some skills or had assumed them to be implicit in the role.
Whereas other companies were more precise over how they described the skills that a job needed.
We could learn from the aggregate data to come to a common skills description of a job with a given title. It would fill in some of the gaps left by the individual postings. It would also allow us to capture phrases and synonyms for skills that different companies had used.
We could also start to pick out the most frequent skills for a job and use that to give an indication of the importance of a skill for each job title.
Step 2: Establishing a jobs network
With a common view of skills for each job title we could now look at how similar jobs are in terms of skills.
<figure><img alt="The overlap in skills between a Software Engineer and a Data Scientist — represented as a Venn diagram." src="https://cdn-images-1.medium.com/max/970/0*HHlhpdVtFjMHivCt"><figcaption>A Venn diagram showing the similarity between two related job titles</figcaption></figure>We constructed a similarity metric between job titles, by comparing their skill sets.
We can do this by computing the <a href="https://en.wikipedia.org/wiki/Jaccard_index">Jaccard Index</a> between each pair of job titles.
This is then used to assign a weight to the edge between a network of job titles. This can be visualised in the following example of Job Titles Network.
<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1dP9ZcVG_xjuZapz"><figcaption>A network of job titles with associated skills similarity measures</figcaption></figure>Step 3: Clustering a network of Job Titles
Having obtained a job titles network we are in the exciting position to solve the problem of having lots of very differently described jobs with very similar skills. These naturally cluster together and for a certain granularity we can create a higher or lower abstraction on the jobs graph.
<figure><img alt="Clustering a graph" src="https://cdn-images-1.medium.com/max/440/0*HPvRN4XnJFfBX0w4"><figcaption>A clustered graph of job titles</figcaption></figure>With this clustered representation we can have a single representation of a set of job titles with a set of similar skills.
Step 4: Ranking job clusters
Finally, we modified the Page and Brin algorithm from 1999 and created our own ranking algorithm across the skills-jobs network using a discrete-time Markov Chain. Now we can take into account how important skills are for a given job (how frequently they are mentioned) — and extend that to clusters of similar jobs.
Now when an applicant lights up some of the skills in the original data set, we are able to produce a probabilistic rank of jobs clusters and produce meaningful importance ranked output based on continuously-updated historical data.
What about algorithm bias?
We need to address the issue of potential bias in the algorithm. This is a massive topic for any recruitment or applicant matching algorithm. While this approach is not immune, the good news is that because the training data is not based on individual applicants we side-step the biggest single source of bias in job-matching — that of prior applicant selection bias.
So where can bias occur? Well two main areas:
<ol>
<li>The job descriptions themselves might contain biased perspective on skills, but unless this is a sector-wide problem, individual posting will get amalgamated with other jobs from the same sector and the effect of the bias will be significantly diluted if not removed altogether.</li>
<li>The candidate applying may be self-selective about how they describe their own abilities. However the algorithm looks for jobs that match partially and “fills in the blanks” for missing skills. This then gets displayed in the percentage match metric.</li>
</ol>
Summary
We have presented a brief overview of our approach to tackling the job search problem. We had a lot of fun doing it and were able to deploy on a GCP-BigQuery-Dataflow architecture two months after starting the project.
If you are interested in joining us for exciting projects like this or just finding out how your company can benefit from our Data Science and Engineering expertise, <a href="mailto:%20info@datasparq.ai">please get in touch</a>. We’ll be delighted to hear from you.
Huge credit goes to the team from Code First Girls who brilliantly innovated on the data science behind the Job Finder Machine with us at <a href="https://datasparq.ai/">DataSparQ</a>: <a href="https://www.linkedin.com/in/erika-nicole-gravina-42a72b176/">Erika Gravina</a>, <a href="https://www.linkedin.com/in/dehaja-senanayake/">Dehaja Senanajaka</a> and <a href="https://www.linkedin.com/in/rajwinderbhatoe/">Rajwinder Bhatoe</a>.
References
[1] The Job Finder Machine: <a href="https://labs.datasparq.ai/job-finder-machine/">https://labs.datasparq.ai/job-finder-machine/</a>
[2] Page, L., Brin, S., Motwani, R., and Winograd, T. <a href="http://ilpubs.stanford.edu:8090/422/1/1999-66.pdf">The PageRank Citation Ranking: Bringing Order to the Web</a>. Technical Report. Stanford InfoLab. 1999.
[3] Oliver Bartlett. <a href="https://medium.com/@oliverbartlett/1a7ffe6aa7ff">Using Data Science to help people get back into work</a>. Medium, 2021.
[4] <a href="https://en.wikipedia.org/wiki/Hierarchical_clustering_of_networks">Hierarchical clustering of networks</a>. Wikipedia, April 2021.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=ad6f7f0fc358" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/job-matching-a-new-challenge-in-search-ad6f7f0fc358">Job matching: a new challenge in search</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Job matching: a new challenge in search

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xSP0_BOiR-iv6tVCXLidOQ.jpeg"><figcaption>But some industries aren’t</figcaption></figure><a href="https://www.datasparq.ai/">DataSparQ</a> recently worked with <a href="http://reed.co.uk/">Reed</a>, <a href="https://codefirstgirls.org.uk/">Code First Girls</a> and <a href="https://cloud.google.com/">Google</a> as part of the <a href="https://emergentalliance.org/">Emergent Alliance</a> to help people get back into work post Covid.
One of the challenges put to the Emergent Alliance by Reed was to assess ways in which job and skills data might be used to help those made redundant during the pandemic get back into work.
<a href="https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/employmentandemployeetypes/bulletins/uklabourmarket/april2021#employment-unemployment-and-economic-inactivity">Figures from the ONS</a> show UK unemployment at 4.9% in April 2021 (up from 4% a year earlier) with hospitality, retail and entertainment sectors especially impacted. Of around 800,000 payroll jobs to disappear since March 2020, 526,000 are from retail and hospitality alone.
What’s more, for some specific sectors like high street retail, it’s not clear how quickly (if ever) employment levels will return to their pre-covid levels.
For those who have been made redundant through this period there are a couple of significant challenges Reed identified:
(a) People whose sector or industry has been significantly disrupted may not know where to look for their next role
and
(b) they may be missing key skills which would allow them to move into other roles or sectors (and may not be aware what those skills are).
We set out to develop a proof of concept (PoC) that would demonstrate how job and skills data can be used to help solve these problems. Head over to the <a href="https://labs.datasparq.ai/job-finder-machine/">Job Finder Machine</a> to have a play.
So what’s going on here? To satisfy challenge (a), we wanted to reimagine job search as a less targeted activity. For example, rather than asking a search system what “Hospitality” jobs were available in your area, the idea was to allow people to tell the service what skills they have, and to have relevant roles suggested back. This way, search results would no longer be constrained to the roles or sectors the jobseeker was familiar with. We also wanted to experiment with the idea of returning generic job titles, rather than specific roles, to see if this could be a useful pre-search tool for job seekers to identify new roles or sectors before using more traditional job search tools.
The underlying data science going on here is much better <a href="https://medium.com/@jeremytbradley/ad6f7f0fc358">explained by Jeremy in his post on the project</a>, but at a high level we analysed thousands of job specifications to build a model which clustered job roles according to their skills. The job roles are returned according to how closely the skills required match the skills entered.
The approach has a number of benefits over a free text search, including:
<ul>
<li>Free text search will only return roles whose description uses the same words as have been entered into the search. Because our approach has learnt from many different descriptions of a role it means we know that a role which asks for a “team player” is likely to be suitable for someone who describes themselves as “collaborative”.</li>
<li>We can reduce the impact of <a href="https://www.totaljobs.com/insidejob/gender-bias-decoder/">gendered terms</a> when either used in job descriptions or as inputted skills. The model understands the similarity between skills and therefore it’s possible to automatically add non gendered (or oppositely gendered) skills to any input to increase the breadth of roles returned (this isn’t yet implemented in the machine).</li>
<li>The underlying model provides a navigable graph which could allow jobseekers to browse roles by skills in order to help them explore further outside of their normal comfort zone. For example, you could imagine for any roles presented back to the user, there could be another 5 roles which require 1 additional skill. Clicking on one of these would open up another 5 roles and so on.</li>
</ul>
The skills gaps problem (b) is something we didn’t have time to implement in the proof of concept, but the approach supports it. For example, it would be relatively easy to identify the top roles that require 1,2 or 3 more skills in order to match the user’s input, and then display those jobs in the results along with highlighted skills gaps.
The PoC has demonstrated that a relatively simple dataset can produce a data graph which provides a powerful, intuitive way to find jobs, while also opening up the opportunity to develop innovative new capabilities like the gender-bias-reducer or graph browsing.
If you’re interested in discussing how the work done here might support challenges you’re facing with recruitment, then please <a href="https://www.datasparq.ai/contact">get in touch</a>, or you can contact the Emergent Alliance through <a href="https://emergentalliance.org/products-and-services/job-finder-machine/">the feedback form on this page</a>.
About: The <a href="https://emergentalliance.org/">Emergent Alliance</a> was set up to aid post Covid Recovery through developing data-driven insight and tools to be made available to businesses and government. <a href="https://datasparq.ai/">DataSparQ</a> contributed engineering, data science and product management effort towards the <a href="https://emergentalliance.org/challenge-themes/labour-market-and-how-we-work/regional-re-skill-redeployment-challenge-statement/">Regional Re-skill and Redeployment challenge</a> submitted by <a href="https://www.reed.co.uk/">Reed</a>.
<img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=1a7ffe6aa7ff" width="1" height="1" alt=""><hr>
<a href="https://medium.com/datasparq-technology/using-data-science-to-help-people-get-back-into-work-1a7ffe6aa7ff">Using Data Science to help people get back into work</a> was originally published in <a href="https://medium.com/datasparq-technology">Datasparq Technology</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Who we are

What we do

Work we've done

Contact us

Jobs

Inspired? Join our team

Datasparq people are smart, curious, skilled, experienced and creative. If you have what it takes to create data solutions that aren’t like everyone else’s, we’d love to hear from you.

Data Engineering

Senior Data Engineer