AI and ML in Drone Navigation: Insights for Product Development

Hi everyone, hope everybody's refreshed. So when I found out I was doing this talk because Manna tend to go out and do a lot of talks, our marketing team told us it was about AI and ML and that they tried to put as many buzzwords into a talk as possible. So I apologize that this talk is actually just about data which, if we all know, is really about what AI and machine learning is.
What I'm going to do is first of all tell you a little bit about Manna just in case you don't know who we are so you have some context, and then I'm just going to show you what we're trying to do with data and AI, and then some practical things that we've done immediately and what we're looking to do in the future.
For those of you who haven't heard about Manna, we are on a mission to improve the world by making Lightning Fast Suburban deliveries green and affordable. What does that mean? That means that when you typically get your takeaway by a road-based vehicle using up carbon emissions, we will now do that delivery by drone which means we don't have any traffic, we're fully battery powered and we're very green and society friendly.
How does it work? It works the same as your normal delivery. You go onto your aggregator app like Just Eat or Deliveroo, you shop, you decide what you want, you put in your address, you pay for the goods. The only difference is that instead of it being late and cold, it's fast, it's hot and it's quick. And that's really it. So we have all the normal user flows that other businesses would have, replicated some of the same technology just to prove out the market fit for drone technology.
Why drone technology will win in this space? It's just faster and a much much better quality product. So our typical delivery will be about 8 minutes 43 seconds from when the customer actually places their order. The actual delivery drone part of that is about 2 minutes average 48 seconds. It's gone up a little bit since this slide because we're delivering a further radius.
So that's what Manna do. We're here, we're live in Dublin at the moment. Probably it is the largest drone delivery operation in all of Europe, probably in the United States as well. Australia is the only country in the world with more operations at this moment in time.
We really wanted to have a drone here today to show you and we just didn't have any spare. But when people think about Manna, the first thing they say is "oh the drone, that's the entire business, that's what everybody is focused on" and it is, it's the cool part of our business. If there's any software people in the room, which I think UX and DX is a lot of software, it's very hard to get excited about software because it's intangible. And as soon as somebody puts in a fingerprint scanner or some physical bit of hardware, as simple as it may be, everybody gets really excited and we're no different.
So we always try and say to people look, the drone is a tiny part of our business. It's actually much more about the applications, the system and the data that we actually work on. But to cover the drone - what we have is our drone is reasonably large. So most people typically think of drones in and around this size. Our drone is about 1 and a half meters all the way around and it's built for redundancy. It's not the prettiest drone in the world, it's a workhorse. So it flies 12 hours a day doing multiple deliveries an hour, and it's designed with redundancy in mind.
So it allows us to do a sub five minute delivery. We cruise at about 85 km an hour so super quick. We can go as the crow flies so we don't get stuck in traffic as I mentioned. We're green, and most of all we're very very safe. You'll see, and this will become important later on, on the drone we've got propellers on every arm, we've got two engines, and we can afford to lose any engine on any arm and we can still fly and that's part of our safety case. Internally we've got lots of triplex systems to keep us safe, but that is not what today is about.
This is again the largest drone operation, delivery operation in Europe. It's down the road in Blanchardstown in D15. There we are operating deliveries to 45,000 households, Tuesday to Sunday 9:00 in the morning till 9:00 at night. We've got about 25 restaurants there. We've just opened a second base on the outskirts of D15 to expand our delivery radius. So if you go down there, have a look in the sky and you should see us flying.
And this is the area and this is what it looks like if we overlay all of the drone routes that we've ever flown. Each orange marker there, each orange line is a flight that we've actually taken and done a real delivery. We to date, I think we're about 11 and a half thousand deliveries this year alone in Blanchardstown. So again, customers love it, really really popular service.
Now data - so how do we use data? You might recognize that's actually an operation we have in the US I should say - it's not a very Irish looking home, that's not in Blanchardstown.
So data, and I think if anybody leaves here today with one thing to look at, it's a tool that we use for data called Metabase and it is very very very useful, very simple, very useful. But if you take one thing away from today it's Google Metabase and see how it can help your business. If anybody in your business says that they need to build a custom dashboard, ask that person to take lunch, go Google Metabase, come back and have it running.
With all of our flights we've done about 200,000 flights. With every flight we get three log files because we've got three flight controllers on board and that's about 50 Megs of data automatically uploaded to the cloud and we process it. Normally we dump it into a database and then we run all of these fancy SQL queries on it to see if we are within tolerance for what we wanted to be achieving in that mission. Our view into that is Metabase. All of these are custom SQL queries. This is one particular flight in the top right hand side for you guys, left hand side for you guys you can see some metrics which I won't bore you with but it essentially tells us that the flight is healthy, what altitude it flew at, how much energy it used and things like that.
So this is something that we've been doing for five years. We've been using these tools and we've built upon them and nobody in this room I think would call that machine learning or AI. That's just basic stuff that we should all be doing with our data.
What we did about a year ago is we said right, we are going to be scaling out our fleet and just to give you an example if we wanted to cover the UK in drones we would need 49,000 drones. So quite a lot of drones and this type of data won't scale well for that problem for a few reasons. One, it's very complex data and there's nuances in what is an anomaly and what is not an anomaly.
So about 12 months ago we hired one guy, Tung, to come in and play with our data and see what he could find and we had great plans. We said yes, we're going to use machine learning for all of our data and we're automatically going to anomaly detect every problem we ever thought we could have. And what we first quickly realized was as soon as Tung started looking at it we hadn't labeled any of our data, which is a big problem.
So the number one problem is you don't record the data in the first place. The second problem very closely followed is you don't label the data which means your data scientists have a very very hard job of trying to figure out what the data does.
So that was the first problem. What Tung did is he ran us through unsupervised learning and he figured out some plots and we grouped our flights and we went to our engineers and we said take this group and this group and tell us is it a good group or a bad group. And we identified these purple dots here and this particular plot is around vibration, which is an important factor for us.
And we said okay that's great we can detect the flights and everything like that. We discovered that this can detect some anomalies but it was giving us a lot of false positives and what that meant was that it was only so useful to us.
So what we ended up doing was figuring out that actually one large model is not a good use case for us and that's because wind has a big impact on vibration. If we take that example or voltage and things like that, so what we've settled on is a lot of small specialized models which means that we don't have to have as much trained data or labeled data because we're focused - each model is focused on an individual part of the problem. So that's our first learning.
Our second learning is that we can actually use these models to great effect in our operation. So if you can imagine our drone operation, we have four drones on four pads. An order comes in, the restaurant brings the food to us, we're collocated with the restaurants, one of our operators is the loader takes the bag and puts it into the aircraft.
Now before every flight until we studied this data and actually understood the problem, this is what the loader had to do. What's happening here is they're doing a pre-flight inspection. Now they slowed this down a little for the demo or for the video today. What they're doing is they're checking bolts underneath on the motors underneath and they're rubbing their hands along the props to look for cracks or looseness in the props and this is something that we've committed to doing again to have a safe operation.
And what happens - our props are folding props and what can happen is the washers in there can get worn out and the props begin to shake and they're not as efficient as they otherwise should be. So this takes let's say 30 to 40 seconds every flight. I promised the guy doing this I would show it all.
So think about this time for every order we get this is the process we're doing. So we went to the data and we said what can we do to solve this and this is still one guy Tung working away looking at our data to understand. And what he figured out is if we look at the two seconds before takeoff just as the propellers are starting to spin up, they've got a different vibration profile if they're ready to go, if they're tight, or if they're a bit loose they've got a different vibration profile.
So we plotted that data on this bell curve and what you can see is the blue here indicates that we've got an issue, the orange indicates we don't have an issue. Now a bit clearer in his violin distribution - so if we look at it by day, the larger the base and the higher the spike, the more vibration which means we need to change the props and we've got tolerances.
So this was great data but we couldn't collaborate it until we started bringing in our labeled data which is our maintenance records and this was only something that we started to really digitize properly in a programmatic way again about eight months ago. So these numbers here are different work orders that were carried out on the craft. So every time an aircraft is touched we actually have an entry that we fill out and we explain what parts were affected, who did the work etc.
So what Tung was able to do is he was able to correlate motor changes with decreases in vibration. So you can see on the second we changed one motor and it went down, fixed another motor it went down, and then we came all the way down into our tolerance.
So what this meant is that we have a proof point now for understanding what vibration is good and bad. So we now do not need a pre-flight check which means we've saved based on our current flight numbers 41 hours a month from a human's time which makes our operation more efficient and the biggest challenge with road-based or last mile delivery in general is the human cost. That's one example where a data model, an AI model has really kind of helped us in a practical way where it hasn't just given us results that we can't really action.
So that's what we like. The next problem we had that we kind of solved with some data is actually our drone routing. So what we used to do until very recently with our drone routing is when you fly you need to reserve airspace. So if you've got two drones in the sky, they're not going to collide. So we have a virtual grid that we make of the sky based on aerial surveys and population densities and that we figure out where to fly with different algorithms.
What we have here is in the old picture we have how we used to do it, which was if you see these yellow boxes or hexagons we used to actually reserve the entire route which means that when we were flying other drones we would have that in our grid and we could route around them - very very simple, very very effective. What that meant though was that we weren't clearing that airspace until the drone came home. We go out and back the same route at the moment. So that limited our capacity in the air which really meant we can only fit about five drones in the sky at one time which on a Saturday night when everybody wants their takeaway is not very efficient.
So we have recently moved to this new system which we call 4D, four-dimensional, and what you can see here is the green is a sliding window so in time space and distance. The white dots are the drone and they need to be there. So what we do is we guarantee where the drone will be inside that green worm or our sliding buffer and that allows us to have crisscrossing routes. We're not constrained by time which gives us basically infinite capacity in the sky.
And that's all very easy to say, it's harder to do and the main reason it's harder to do is what Irish people love to talk about all the time is weather. So wind has a big big big impact on drones and anything in the sky really and you know I personally didn't appreciate how difficult a job the weather forecasters have until we actually hit this problem.
And what happens is if we're flying into a headwind it will really affect the performance of the craft. Likewise if we've got a tailwind it will make it go faster which in our 4D model will push us outside containment. So we said okay simple, we'll put up a weather station which we have already because we use that to determine if we can fly or not fly, but it turns out our local weather station wasn't accurate enough because it was being shielded by buildings and things like that.
And what we later found out is actually no weather station is accurate enough because they'll always skew their data. So what we did is we took three different types of weather sources and one of them, the external one which we validated is a 10-minute weather source. Our current on base weather source was I think what do I have here - 1 minute exact by 5 seconds.
And what we did is we needed at least a one minute but we couldn't get that from an external reliable data source. So what we did is we plugged all of our data into a model and we formed a forecasting model. So what you can see here is the blue dots here are the external source of data which we get every 10 minutes which is why we've got flat lines and then the other ones are more real-time data but you can see they sort of match sometimes depending on wind direction and things like that.
So it's not an exact science which is where the model has really helped us. So what you can see here is our predicted weather value which is the orange based on the actual different weather sources that were coming from and what this does is this allows us to maintain our ability in that column which means we can guarantee that we can stay in tolerance of our 4D window which again as I said means that we can have unlimited craft in the sky at any one time.
I think what I wanted to show today was just a bit about how we're using data practically. We've got big plans for how we want to use these AI models and we're getting better and better at labeling data. I mean I think it's a slow process, everything's a slow process when you're doing it for the first time.
I think something that we intend to have in place by this time next year is an aggregated model of all of these smaller models so that when we do scale our drone fleet to thousands we've actually got a single health check, a fleet check to be able to verify what we're doing.
So what's next for Manna in terms of our data journey? We're going to develop more and more smaller models to pick up things that otherwise we would miss, keep us safe. We're going to refine our models to enable preventative maintenance. So the MTBF minimum breakage time for a part or for a fault is a key stat for us, and we believe AI can help us do that as long as we're maintaining our work order records correctly so we can programmatically use those in our models.
And then as I said there we want to scale these models. So hopefully if I come back next year - or you're not in Dublin next year we have to get on to worry about that - or get a trip to New York. So the next time we give this talk we intend to have prediction maintenance. So not only will we detect that our props are loose and stop flying, we will actually be able to tell our maintenance team two weeks in advance so they can actually schedule maintenance and not react to maintenance and I think that's the power of data for us.
A great thing one of our advisers always reminds us of is "all models are wrong but some are useful" and I was able to find on Google this was attributed to George Box and it's one of the great things that if you're working with data scientists and that are looking for perfect - there is no perfect, perfect doesn't exist. So I would always remind them that all models are wrong but then some become useful.
[Q&A Session]
Audience: How do you manage route planning if competitors enter the market and are there regulations on sharing flight routes?
Alan: Great question, somebody's been thinking. Yeah it's one of the big challenges that the drone industry has is how do you cooperate and share airspace. One thing we recently did with one of our competitors but also one of our friends in the industry is Wing, which is Google's drone company, and we demonstrated in the US Wing drones and Manna drones actually collaborating and swapping. It's called UTM, unmanned traffic management, which is a bit like aviation's air traffic control.
But the expectation is that there will be 10x more drones than there are manned aircraft so it can't be any manual process. So the industry has come together and developed UTM standards, APIs basically, and governance policies. So when we go to plan a route we can share our route with the industry, they share their routes with us and we deconflict and that's where the 4D comes in. And that's why you need to be able to predict where you're going to be because if you're sharing the airspace with other people they need to know where you're going to be and you can't just do it in real time.
Audience: Why should sky be designated to commercial deliveries and are we looking at the end of a blue clear sky? What about the users that are not ordering stuff at the moment in time?
Alan: I mean I think it's fair question. I think you know there's loads of benefits, societal benefits like the environment things like that. I think do people feel the same way about road-based deliveries that they're not getting? Do we like sitting behind Amazon trucks in traffic and delivery bikes? I mean what we do know from the data is that cyclist and road-based delivery drivers have a very very catastrophic incident rate in terms of when they're trying to make deliveries around the world. There's some frightening stats on that.
Audience: How much cheaper are drone deliveries compared to a person on a bicycle?
Alan: Good question. So I mean not only is our business exciting and cool and all of that stuff, it's actually very practical. So if you look at all of the current delivery companies that are delivering food they actually lose money on nearly all of their deliveries, which is why the cost of the takeaway is so high because people are trying to make the margin somewhere else. Drone delivery - we're already lower than the cost of road-based delivery to restaurants.
Audience: Do you collect your data in real time?
Alan: Yes we collect real time data from our aircraft. I suspect the question is more do we collect environmental data and no is the answer. We have a camera on board but that is only used at delivery to verify the delivery zone. There's no collecting of data, that's something we're very conscious about.
Audience: How did the company find product market fit?
Alan: We did it with a lot of work. I mean, so we originally the business started as a B2B business where we would power other people's deliveries like restaurants and things like that. And to actually prove market fit, we actually had to create a store and create an app and create all of these things just to prove the actual technology. So we're currently running a Manna app at the moment in Blanchardstown, we're integrating into a few other third parties to be a B2B delivery option but we currently have all of the headache and systems that come with managing actual customer orders.
Audience: What percentage of time are you unable to fly due to the weather in Ireland?
Alan: 3%. We can deal with 97% of Irish weather. It's pretty good. Yeah, we wouldn't be flying in Florida right now.
Audience: What was the impact of the weather forecasting model that led to 4D versus the old route planning and how did it impact the business? Was there an increase in safety?
Alan: So there was an increase in safety because we were working on a model where we blocked the entire route so there was no impact on the wind. So we only moved to this 4D model once we were confident in the weather model that we had. So there was no impact on safety. There was a massive impact in the capacity that the sky now has and it gives us much more flexibility in terms of how we route in that we can fly out one way now and back a different direction.
Audience: What was the weight limit for the drone and can I get a couch delivered?
Alan: You can get a very small couch. The maximum takeoff weight for the drone is 25 kilos. At the moment we deliver 2 and a half kilos of payload. That's pretty good. Like we designed it that way and the next iteration of the craft will actually have a three and a half kg payload capacity and that's really what Tesco's called the emergency shop where people do a single basket shop during the week - is three and a half kilos.
Audience: Did you interview birds and ask for permission to fly into space?
Alan: No we nearly did. In our operation in Balbriggan where we were for two years before Blanchardstown we actually had a set of nesting seagulls beside us which liked to attack us every now and again but they love the drone - they didn't like the humans.
Audience: I remember this use case in Dubai I think they've been practicing this - are you going to transport people with your drones, AKA a drone taxi?
Alan: Yeah I mean that's a whole other ball game. Urban Air Mobility - there's a lot of companies working on that and doing a good job. It's a massive cash burn as an industry. Once you involve carrying humans it becomes very very complex. A lot of the technologies that we're building like the air traffic management, the UTM technology is feeding up into that - air taxis, but it's not something Manna are looking at.
Audience: What does the future look like for residents - noise nuisance and privacy?
Alan: Yeah I mean I think you know a lot of people have a bad impression of drones because they think about the hobby drones that are bought in the different electronic shops that have a quite a high pitched kind of annoying buzzing sound. So I mean our drone again it's about a meter and a half, the props are about 60 in kind of 32 in at either side and they actually have it's more of a hum.
So when our drone is in the sky you don't hear it. You hear it when it comes into a delivery but it's actually quieter than a delivery van coming to your house and it's there for far less time. So it's you know it's a new thing and we're very - when we move into a community we do a lot of community engagement. We bring the drone to schools, we do evenings for the residents to kind of show them the technology and how it works and that allays a lot of fears.
But I mean we appreciate that new things can be scary. I mean typically when we go into a new location everybody looks at the sky for three weeks and then it's normal. People just love it and they love that they can get their coffee or their ice cream or whatever they want.
Audience: How do you incentivize people to actually purchase the order from not from another kind of order delivery?
Alan: I mean there's no real incentive needed. It's the same price as a road based delivery, it's just quicker. People very often - people very quickly forget you know that getting quicker is a new thing and it - like really they order from us on repeat for the quality. And it's in new categories as well so we actually run a coffee shop because coffee is a huge item for delivery and I think if any - if you said to anybody do you want a road based coffee they'd laugh at you. So it's opening up some new categories that we didn't realize at the start.
Audience: Are some people so disagree with drones that they shoot them? Do you have a lot of loss?
Alan: We've got no loss. So our drones - I should have said earlier on - our drones don't actually land so they don't come near humans or the customers. We typically fly at about 65 meters and when we get to your home we descend to about 15 meters off the ground and underneath there's two doors that open and the payload comes down on a thread. And again coffees are one of our number one items and we don't spill a drop.
We are now in Texas and we've had one person come to us and say they're going to shoot us down, and it turned out they were joking which is good.