Your first 10 steps in the Robotics Journey
What to expect when you’re expecting a robot?… Buy robot, build solution, take over pizza industry, right?
But robotics is an emerging, constantly changing field. It’s a journey with unique challenges. In this blog post we want to provide our take on the steps in the journey and how you set yourself up for success.
We’ll use this as a roadmap for upcoming blog posts, YouTube videos, and in Jim’s Random Walks livestream, so subscribe to keep up to date.
Steps and challenges
Start: The robotics journey
Whether you’re disrupting an established industry with an exciting automation project, or you’re a hobbyist exploring the latest technology, welcome!
Challenge: There’s a wealth of information, libraries, and academic papers on robotics… Where to start?
Solution: Let’s map it out, and we’ll discuss each step (and pitfalls) in the coming months.
1. Hardware Choice
The first step is generally choosing hardware. This is the device that’s going to solve the physical world problem: anything from a simple cellphone, through a tracked vehicle, to a submersible. Luckily, there are endless hardware options to choose from across a wide price range! More to follow on this topic.
Challenge: You need to pick a hardware platform that will solve a physical world problem.
Solution: Pick a hardware platform that matches your needs, or build it!
2. Sensor Choice
Next you’d like to give it sufficient sensors to be able to solve the problem at hand. There’s a variety of sensors – cameras, LIDARs, RADARs, compasses, and many more – picking a set that should solve the problem (given a budget, some of this stuff can get pricey!) is the next hurdle you need to overcome.
Challenge: You want to give the robotic equipment (a.k.a. robot) sufficient information to understand its environment. We call it the Sensor Goldilocks problem – not too much, not too little, just the right amount of sensor data.
Solution: You choose which sensors you want to use for your application. This requires some guessing because you don’t know what you’re going to need in the next steps. We’ll talk through some options and how you can confidently choose sensors that will work in the next steps.
3. Integrating Data
Now we switch from hardware to software. You need to bring all your sensor data and your actuation devices into one place so that you can start processing it and building the software logic.
Challenge: You want to integrate your sensors to start understanding your environment. How do you consolidate the raw data from all your sensors, say a LIDAR, a camera, a compass, and a GPS? Choose carefully, because we’ve often made poor choices here and ended up having to write our own device drivers. Pick correctly and integration is a breeze.
Solution: At this point you’re probably deciding whether or not to use ROS (Robotic Operating System), which version to use and what packages it supports out the box. We’ll talk more about these design considerations in this step.
4. Building a Map
Before the robot can do truly useful actions, it needs a robust understanding of its environment. This is called perception (a.k.a. SLAM, mapping, spatial awareness, etc.) and it is the critical step where disparate raw data is converted into information. This is a rapidly developing research area (funny story: reproducing a human’s spatial awareness is quite challenging) and allowing you to do it easily, robustly, and in a scalable way is NavAbility’s mission.
Challenge: Converting camera images, LIDAR scans, compass bearing, GPS location, beacons, etc. into one robust, consolidated map of the world. Assume that you’re also going to have imperfect information and will need to design for that – this is true for every scenario we’ve ever worked with.
Solution: Start reading up on SLAM, dig into an existing library, or use our cloud services to get going quickly.
5. Using Landmarks
Landmarks are identifiable features that contain information and help localize the robot, like a docking station, a tree, or a cup. Converting raw data into landmarks is critical in closing the loop for robotics and solves the critical challenges like: “I’ve seen this before, so I must be here.” Converting raw data (like camera data) into robust landmarks and relating those landmarks to your current position is an important part of robotics.
Challenge: How do you convert raw sensor data into robust information so you can identify landmarks (known information, or objectives, in a map)? A great use-case of this is finding a docking station in a room.
Solutions: There are great libraries that help convert sensor data into information, and we’ll discuss these in regular video posts. For example, the YouTube video on using AprilTag fiducial markers to convert raw camera data into real-world landmarks.
From research to product!
Great! Once you have proven out the idea, you need to take it to the next level.
These steps might occur in any order, but we’ve documented the journey in the sequence we find users like to implement them.
6. Adding Memory
Once you flip the off switch, you don’t want to lose your data. On Monday I turned the robot off. On Tuesday I have to start over. I want to reuse yesterday’s information. Persistence is key to real-world robotics, but it’s quite a challenge because saving logs won’t cut it outside of a lab.
Challenge: How do you transmit, save, query, and visualize your robots data over time? Furthermore, how you give it yesterday’s map to use as prior information for today?
Solution: Integrate a persistence layer that saves and indexes your robot’s data, both temporally as well as spatially. You build multiple data sessions and use yesterday’s information as prior data to improve today’s navigation.
7. Using Prior Data
Prior data was mentioned in persistence, but what if I want to include blueprints, CAD models, or known locations? This is invaluable in maps where you’re doing construction, navigating a congested harbor, or finding a box in a warehouse. Luckily, you already have a persistence layer, so the challenge is to represent this information as dynamic landmarks.
Challenge: How do I convert prior information (blueprints, CAD models, Google Maps, known AprilTag positions) to landmarks so that I can use it for navigation?
Solution: Design a data integration layer between your prior data (e.g. CAD model) and your map so that the prior data becomes persisted memory as dynamic landmarks.
8. Cooperating Robots
Multiple robots can share a map so that they can operate together. This is the vision: coordinated, cooperating robots solving real-world problems!
Challenge: You have many robots operating and want to them to share information and coordinate operations. How do you share information, build a common map, and coordinate actions?
Solution: You consolidate each robot’s sensor data into a global map and stream information to each robot from the global map. Each robot then has a small local map and a much larger global map in the cloud! We live, eat, and dream about this, so we’ll discuss this in many forthcoming posts.
9. Many Robots, Many Maps
As you add new environments (new maps) your data grows exponentially. Managing this data and using it between robots becomes an enterprise challenge.
Challenge: You have many robots operating in a variety of environments. How do you store, query, and leverage all of that information?
Solution: Each environment becomes a shared global map, which grow as robots explore or interact with it. Environments become living global memory for any robot that interacts with it.
10. Onward Robotics Journey!
To #INF and beyond!
10. Onward Robotics Journey!
That’s the first 10 steps in the robotics journey!
Feel free to reach out to us if you’re on this road, we would like to understand how you’re solving these problems and discuss how we can help you move faster.
The full map of the Robotics Journey
We’ve compiled this as a mini-map with notes and considerations. Please feel free to download and print out, or use in discussions.
How can we help?
We want to help you in your journey. Where are you on the roadmap? What is your minimum viable navigation solution? What exciting projects are you working on that are over-budget, not getting to market, or may be cancelled because of pitfalls like these?
