AI-Driven Innovation: Transforming Retail & Fleet Management for the Future

Pritam Roy at UXDX Community: The AI Edge: Innovation in Action. Video: https://youtu.be/2uuTQG0yXBc

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

AI in fleet management

[00:00:08] Thank you so much for this introduction. To give a brief about me, I have around 20 years' experience working in this industry, and today I will talk about the usage of AI and machine learning in fleet management. The fleet management industry is always evolving. Back in 2019, there was a paper published by Volvo on how they could capture telematics to improve the working of the machines. Since then, different industries in logistics and transportation took those learnings and different research papers and started to evolve, using fleet management systems in their companies.

[00:00:51] For the transportation and logistics companies, the integration of AI into fleet management opened new possibilities in this era. The business is supposed to increase from 8.1 billion currently to 19.5 billion by 2032 through the usage of AI technologies. The highlights are the use of machine learning, IoT and cloud computing devices to enable predictive maintenance and intelligent routing optimization.

[00:01:27] Think about how most people today use Google Maps to travel from one place to another. What if these things were used by a driver to deliver a product to a store? Suppose a store sends an SOS that they are running out of stock for a specific product during a promotion or during the season. A driver can plan his route and predict when he will reach the store, and that way it can be fulfilled for the end customer, by the company or by the store. It really opened new possibilities. Stores can share their warehouses with different customers, different kinds of people, and route planning makes all these things possible.

Predictive maintenance and fuel efficiency

[00:02:10] To summarize how we can use AI in fleet management, let's divide it into a few comprehensive steps. One is predictive maintenance, where AI systems can forecast vehicle failures with up to 90% accuracy, reducing breakdowns and maintenance costs significantly. Imagine a vehicle which is going to break down. If you can predict that, it will save the company a lot of money, because they can go and fix that vehicle, rather than having a vehicle driver calling them from the middle of a road at night saying, "OK, my vehicle has broken down. I need a mechanic to be sent here," or "My vehicle needs to be towed." That has helped companies a lot in predictive maintenance.

[00:02:55] On fuel efficiency: this will definitely help with fuel efficiency, because using AI we can predict driver behavior and how much fuel he uses in traffic. Future directions in this are that emerging technologies such as federated reinforcement learning, blockchains and EV-specific analytics can further enhance our systems for data security and fleet maintenance.

Capturing telematics data

[00:03:21] Now let's look at the diagram here, at how we can capture the data that can be utilized for maintenance. Imagine that on the left side of the diagram we have a driver who has a telemetry device installed in his vehicle, which sends data to different data providers. We will have different telematics pollers capturing the data from different streams. We can route that to Kafka. We're using Kafka as a data stream because it is very fast; it can support around 100k streams per second.

[00:04:00] We can route that to different databases, which can be in-memory or fast databases like Kinetica, maybe MemSQL, which can aggregate the data in real time, and then create some dashboards which can be controlled by a provider sitting in his office to see, "OK, where is my truck? What's my location? Is there any storm coming? Can I suggest to the driver an optimized route to reach the destination?"

[00:04:32] Apart from that, it will also send that office a remote diagnostic of the vehicle. How is my vehicle performing at this location? Do I see any issues in the brakes or the tires? Do I need to ask the driver to stop at a certain point so that I can fix the issue? That will come from the operations hub. Apart from that, once a day has passed, the data can be sent to an analytics hub, where data scientists can use it to see how the driver behaved while driving. Did he drive fast when there was a speed limit? Is he using a different route from the planned route, which is delaying my delivery? All these things can be captured in fleet management, for better user behavior and for the good of the company.

The conceptual model and the real-time architecture

[00:05:21] Let's look at the conceptual model and what we can do. Suppose there's a provider which leases its vehicles. When we lease a vehicle, there are certain steps. One is that we go for an agreement. We can buy certain products. That vehicle gets attached to a supplier for a certain time period, and we also enter agreements with certain customers. All this can be controlled in this data model flow. The data will be in one location, where it can be used for predictive maintenance.

[00:05:57] How will the real-time architecture look? There are certain external data providers which will send us data. We have Zonar; we all know about it. We have Samsara, which is also a provider. Verizon Connect, Cortex, Geotab, Trimble, Omnitracs. All these providers today are sending data to different industries, collected from the vehicles.

[00:06:26] We will install a device in a vehicle, maybe in the fuel tank, or maybe we can put a sensor in the speedometer. Maybe we can check the tire pressure by putting in a device. Maybe we can check battery health. This data will be pulled and sent to a Kafka topic. Different applications will have different topics. We will have a reading application which reads the data from different topics, and then, using a data flow in an ETL tool, we can send the data to a data lake, where it will be stored for history purposes, and into a data warehouse, which will be used for analytics purposes.

[00:07:04] So we have different external data providers which support us with the polling devices, like Zonar, Samsara, Verizon Connect, Cortex, Geotab, Trimble, Omnitracs and Volvo. We can pull all this data using a Kafka topic and send it to applications. Real-time applications will manage things like spatial data in MemSQL, maybe a Kinetica database. It can be sent to Databricks, or maybe Snowflake, and we can have a data lake built over S3, which can be used for analytics purposes. There are certain tools in Amazon, and of course our product is mainly in Amazon, so I have mentioned Amazon SageMaker, which can be used for analytics purposes.

[00:07:56] All this data that we are sending from telematics, like the data from the battery, from the engine, fuel consumption, tire pressure, the brake system and battery health, can be used by data scientists to predict whether my fleet is performing perfectly, whether the vehicles are following the proper route, and whether there are issues in the vehicle. That can be used by the end users or by the company managers to see what my route optimization was.

APIs for location, recovery and proactive maintenance

[00:08:25] In the next slide, I will talk about the APIs which we generally use in managing those vehicles. On the left side, I have a service called location management, which will hit the API gateway to get the data and the real-time location of the vehicle: where my vehicle is located, how it is behaving, whether it's following the route. We can use a Markov analysis algorithm for it, which will state what the previous spot of my driver was and what the next spot is, so that I can predict where my vehicle is going. If I can get traffic data in real time, I can say how much time it will take to get to a warehouse.

[00:09:07] We can also have a vehicle finder API to locate the vehicle at a specific time at a specific location. If my vehicle gets stolen, this can possibly be used by different agencies, federal agencies, to locate my vehicle. That is a good use case; these days we see federal agencies locating vehicles that are stolen in the US. Apart from that, suppose a rental company is renting out their vehicles. They can see in real time where the vehicle was taken. If somebody is hiding the vehicle, it can be tracked, and we can get that vehicle back without any difficulty.

[00:09:49] There is also an API that we have developed for proactive maintenance. Based on the pings, we can send an API request to the vehicle asking it to return to the nearest service center. The driver will see that on the dashboard, through which he can take corrective action and return it to the nearest service center. All this can be managed at the back end in different kinds of databases. I have mentioned geofencing: unique location, actual location, not reported location. This is where the vehicle goes somewhere we have not planned. We can store that in the database to warn the driver when he has gone off track or off the road with the vehicle. Apart from that, we will have a data set for unit location, which will state the current location of the vehicle.

The technology stack

[00:10:46] On the technologies that we generally use: for the CI/CD platform, we can use any generally available technology. Currently we use Bamboo here for CI/CD. We have Bitbucket, we have Maven, we use Flyway, and we have Jira. For deployment we use Amazon EKS, or different in-house libraries. For UIs we use Spring Boot and Spring Security. For containers we generally use Docker containers here.

[00:11:23] Streaming I already mentioned: we use Confluent Kafka topics for streaming purposes. Analytics is a big ball game. We use Amazon EMR, Amazon Athena, Amazon SageMaker and Amazon QuickSight. Dataiku is evolving, and there are certain cases where we can use Dataiku for prediction purposes. For observability, we generally use Wavefront, and we use Logstash. What we do when we get the logs from different vehicles is try to apply analytics to see whether all the components I'm using for prediction are working properly or not.

[00:12:08] We use PagerDuty and SMS notifications to send real-time messages to the driver. In short, with these kinds of applications, what we have done is stabilize the fleet industry. Today, in real time, we can know where my vehicle is. I can plan which route my fleet can take. Apart from that, I can predict shortages at my warehouses and route my vehicles to that place accordingly.

Challenges: investment, data quality and complexity

[00:12:40] To talk more about optimization: there will always be a high initial investment. Generally, industries these days don't want to invest more on this up front, because the cost of sensors, infrastructure and AI platforms can be significant. Imagine that we have put in an LLM to predict on all this data. It will be a huge data set, running a few GPU servers, and it will require significant investment and ongoing cost, and the question is whether a company is interested in doing so. Initially it is a challenge, but in the long term this generally helps the industry significantly in maintaining its fleet.

[00:13:24] The next challenge that we will have is data quality and availability. The data that we get will be huge, certain petabytes per day, and maintaining the quality of the data in these cases is really difficult. We have to rely on a high volume of data at high frequency, and the parameters used for data quality should be significantly high and should be in real time. Generally there'll be a lot of noisy and inconsistent data, which will lead to a lot of false positive cases. To learn about them, we have to go into a reinforcement learning phase, where based on the data set we learn new optimization techniques and implement them again. It will be a huge learning curve. It took us around four years to implement such technology in our current company.

[00:14:18] Then there will be complex implementation. You will need skilled personnel like data scientists and engineers. There'll be a lot of system integrations, because we will have a lot of third parties. Different systems talk differently, and integrating with different systems will be a challenge. Then there'll be training of drivers, and maintenance.

[00:14:38] The AI model, as mentioned earlier, will need a lot of validation to remain accurate. There will always be environment changes. Right now we're talking about diesel vehicles; the future that is coming is electric vehicles. All the sensors that work today with diesel vehicles should also work with electric vehicles, and that will bring significant challenges. So it will be an ever-changing one. Apart from that, with the technology: today we are using one technology, maybe Spark; tomorrow it will be Flink. Evolving along with the technology is also a challenge in fleet management. You always have to be up to date.

Challenges: security, scale and false alarms

[00:15:17] Cyber security risk. What will happen if your data gets hacked? What will happen if one of the sensors gets hacked and turns rogue, sending you wrong data? That will bring down my entire system and my entire prediction system. That also requires robust security protocols to protect sensitive vehicle and operational data. Then there will be scalability challenges. As we are dealing with petabytes of data, we need systems that will support petabytes of data. Ensuring real-time processing and storage at scale will be technically very, very demanding.

[00:15:55] False alarms: overly sensitive models may trigger unnecessary maintenance and increase costs. Imagine that my vehicle is sending a wrong message that it is going to break down. I take it to a workshop, and the serviceman comes and tells me, "No, there's no issue with the vehicle. You got a wrong signal." That will incur significant cost. So we have to be very sure what kind of model we are bringing in and what it will be predicting.

Future scope

[00:16:21] Future scope: in the current state, there are still a lot of things that we can work on to enhance this. We can add sensors for battery voltage, ignition state, maybe engine coolant level. Maybe we can add a sensor to the odometer to see if it's functioning properly. We can put in speed sensors so that the driver maintains the speed: if he's going over the speed limit at a certain signal, it automatically controls the vehicle and slows it down, so the driver can never go over the speed limit.

[00:17:00] Transmission oil: if you're running low on oil, it can indicate to the driver that you're running low on oil and route him to the nearest center where he can fill his vehicle. Or imagine a battery vehicle driving on the road, and the battery is low, and the driver is unaware of that or ignores it. The vehicle will automatically send an alarm to the nearest operations center to ask the driver to route the vehicle to the nearest location where he can charge it; otherwise it will incur significant cost. So in a sense, this industry is evolving, and there are still a lot of things to do. That ends my presentation, and I am open to any questions.

Q&A

[00:17:41] Host: I love that presentation, because it showed the complexity. A lot of times people think, "Oh, this is a fairly simple system," but you showed all of the tooling. I loved as well the cases that you gave of how you have to factor in if something gets hacked, or a faulty sensor. Can you go into a bit more detail? I'm not sure if I missed it or if you explained it: how do your models handle the situation where you might have a faulty sensor or a hacked sensor? You said you have to account for that in your models to make sure it doesn't skew your output. Can you go into a bit more detail about how you actually do that?

[00:18:26] Pritam: Oh, it's a wonderful question. A sensor generally sends me a throughput per second, which we capture in real time. There are around 120 data points from a sensor when it comes to us and we read it. A data point can send you a signal that, "OK, my noise level from this vehicle, the threshold, is maybe at 16.1." I'm just making up the number. The next time it sends me a signal that my noise threshold for my sensor is at around 14.1. That means within one millisecond, the noise attribution changed significantly. That's the reason it is sending me two void signals.

[00:19:17] At one time we can see, "OK, my vehicle deteriorated significantly." But we have history algorithms which will go and check against similar pattern behavior from sensors which were faulty during runtime, whether my current signal matches that frequency or is beyond that threshold level. Through that, I can confirm whether my sensor is behaving properly or something is wrong with it. Then I need to dig deep and see what data points the other sensors are sending.

[00:19:47] If other sensors are also sending different kinds of data sets which look conflicting, I assume that my vehicle's data points have been hacked by some third party for malicious purposes, and I can immediately disconnect that vehicle from my data set, from my Kafka topic. That saves my algorithms from sending a distress signal to the operations center that would make it look like all my vehicles are getting distorted by this noise. Hence, whenever I see these kinds of predictions, based on our threshold values and historical machine learning algorithms, we immediately disconnect it. I hope I've answered your question.

[00:20:23] Host: That's really interesting, how you've gone down to the engineering level to figure that out. On the other complexity, I loved looking at your system architecture, where you've got all the different levels. The first thing was that it scared me; it scared me how much was there. But the second thing was, when I was looking at the data stuff, for example, I saw DynamoDB and Snowflake, and there seemed to be a little bit of... I wouldn't necessarily say duplication, but were all of those points necessary? Why did you decide to introduce different technologies, where potentially you could have streamlined and used a single one?

[00:21:11] Pritam: Oh, it's a very good question. When we talk about data, there's a huge volume of data, and there are certain limitations on which data storage systems we will use. MongoDB is generally very fast, so if I have to do some real-time predictions, it is best suited for that. DynamoDB for batches: although it is very fast, whenever you are getting more than one lakh 50 [?] throughputs per second from a Kafka topic, we have seen the DBs slow down. So generally we use that for batch predictions. If there's something which I can predict after a delay of 15 minutes, I generally use DynamoDB. But whenever there's a near-real-time prediction, we send the data across to MongoDB, which will retrieve my data into my prediction in a few seconds.

[00:22:03] For the other storage systems: whenever there is a delay, like when I run batch jobs weekly or maybe daily, I use them, because they are cost-effective for storing data from a longer-term perspective. That's the reason you see different data stores. It's a kind of optimization between cost and what is required. If I create a large-scale MongoDB system, it will be hugely costly; it's expensive for the company. So we've worked out a combination of which system to use for which purpose, so that we can manage our costs as well.

[00:22:35] Host: Great. So it's fitting your model to your needs. That's great. I just want to thank you once again for sharing, because I thought it was really interesting looking at it from the more technical side. We hear so many talks about "Just integrate GPT or AI into your systems." But I really loved the example that you gave in the answer as well, about how you make sure your model doesn't get polluted, and how you actually keep up with that real-time processing. So I just want to thank you once again, Pritam, for sharing.

[00:23:12] Pritam: Thanks for the chance, and I appreciate the opportunity to speak here. Hopefully I will speak here more. Thank you.