Are banks spending too much money on data science?
Data science might be all the rage these days, but it can also be very costly. At the most recent Abu Dhabi Machine Learning meetup, speakers illustrated the ways firms are spending too much and how they can reduce those costs.
One such speaker was Conor Spicer, formerly a senior data scientist at IBM, now working at AI optimization platform DataRobot, which Spicer says "eight of the top ten US Banks are customers of."
One of the biggest cost savers he noted was the basic approach, driven by whichever data scientists are being hired. Spicer noted that, at one of his former employers, a model was presented to two data scientists: one had a physics PhD and worked on modeling for a number of nuclear reactors, the other came through the graduate program. The two had very different approaches.
"One believed he could build a PyTorch model, went down to some low-level details of that. It took him about three weeks," Spicer said. The other opted for a linear regression model, which "took him about an hour." Given that people are expensive and that the cost of ready-made AI models is $30k every two months, the non-PhD hire was far more effective.
Another factor where talent plays a role is skill gaps, particularly when it comes to software engineering knowledge. Spicer says "the most frustrating thing about a data scientist, particularly one coding in python, is package dependency."
To combat this, companies may rely on infrastructure to do the grunt work for them, be-it using DataRobot or building your own in-house system like banking fintech Revolut.
You also need to know when to let an idea go. Spicer says "north of 50% of valid interesting ideas won't make it anywhere near production," and that models should generally be abandoned if they aren't delivering value in some form by the six-month mark. As the parable of Goldman Sachs' Marcus consumer project has taught us, banks have a tendency to double down on passion projects when the value isn't clear, which is something they'll want to avoid when implementing AI models.
Banks are very large organizations, too, which raises issues over the logistical expansion of their data science efforts. A solution to this may come from another speaker, Oskar Mencer, CEO of Maxeler Technologies. His parent company, Groq, is developing an LPU (Language Processing Unit), to compete with FPGAs, CPUs and GPUs in the hardware space.
Mencer claims its proprietary GroqNode operates using less than half the power of NVIDEA's DGX H100 GPU and less than two thirds the power of the DGX A100. This can have a massive bearing on energy saving costs, however it is specific to natural language processing.
Have a confidential story, tip, or comment you’d like to share? Contact: +44 7537 182250 (SMS, Whatsapp or voicemail). Telegram: @SarahButcher. Click here to fill in our anonymous form, or email editortips@efinancialcareers.com. Signal also available.
Bear with us if you leave a comment at the bottom of this article: all our comments are moderated by human beings. Sometimes these humans might be asleep, or away from their desks, so it may take a while for your comment to appear. Eventually it will – unless it’s offensive or libelous (in which case it won’t.)
