Engineering Spotlight: Robotics and Computer Vision with Rich LeGrand, President of Charmed Labs

Learn about roboticist Rich LeGrand's work with Charmed Labs and CMU CREATE lab and what makes robotics so cool, as well as his thoughts on what makes being an engineer so exciting in today's day and age.

Rich LeGrand is President of Charmed Labs, a company that strives to make advanced technology available to a wider audience through low cost and ease of use.

This past summer, Charmed Labs worked with Carnegie Mellon University's (CMU) CREATE lab to release the Pixy 2 camera, an "all-in-one" computer vision device. The Pixy 2 is the second generation of the Pixy Cam, ultimately the 5th generation of the CMU Cam. 

 

Image courtesy of Rich LeGrand.

 

LeGrand's passion for robotics began at a young age.

I’ve always been interested in computers and mechanical systems, and robotics just... I don’t know, when I first was exposed to it, it just really clicked. I thought, oh wow this is amazing, I can’t imagine anything cooler than that. 

AAC's Chantelle Dubois had a chance to speak to LeGrand about his inspiration to become a roboticist, his work with Charmed Labs and CMU CREATE lab, as well as his thoughts on what makes being an engineer so exciting in today's day and age. 

Chantelle Dubois (AAC): Tell me a bit about yourself: What did you study and where did you go to school?

Rich LeGrand (RL): I’ve always thought of myself as a robotics person. I’ve always been attracted to the problem of robotics, how to make autonomous systems, that kind of thing, and that’s the basis of my background. I went to school and studied computer science and electrical engineering with robotics in mind. Eighth grade is about the time that I really [became interested in] robotics, [and] I’ve been trying to build my skill set ever since then. 

I did my undergrad at Rice University in Houston and I went on to grad school at North Carolina State, and that’s where I did some real focus on robotics. 

AAC: What was it about robotics that piqued your interest?

RL: Good question. I’ve often wondered that. If you have something that you really enjoy, it’s hard to figure out what it is about that thing that makes it so attractive.

I’ve always been interested in computers and mechanical systems, and robotics just, I don’t know, when I first was exposed to it, it just really clicked. I thought, oh wow this is amazing, I can’t imagine anything cooler than that. 

I’ll tell you a quick story: Back in 1982, one of my brother’s friends (who was about five years older than me) was always doing really strange things with electricity. Back then, you would call kids who were good with computers whiz kids, so he was the quintessential whiz kid from the early 80s. One thing that was cool was that whenever I showed up to his house he would show me what he was working on and I really appreciated that.

One day, I went over there and he had taken one of those Big Trak toys, a popular toy that had tank-like treads that you can program to do things, and he had taken it apart and hooked it up to his Atari 800 joystick port. He would move his joystick and then the Big Trak would move in the direction he indicated with his joystick. Relays were clicking, lights were flashing, and after it was all done he would hit the return key and it would repeat what he had just done. [I thought] "whoa, that is crazy I never knew a computer could do that".

 

The 1979 Big Track. Image courtesy of Toys You Had.

 

That was a moment of inspiration, and it was something I wanted to learn more about.

AAC: How did you get involved with the Pixy team?

RL: I got involved through Carnegie Mellon University, where I’ve been on a couple of projects with the Create Lab at CMU, including a project called GigaPan and the Telepresence Robotics Kit (TeRK). 

 

Image courtesy of GigaPan

 

Illah Nourbakhsh, the leader of the lab, has all these great ideas of how to get technology in front of different audiences which usually have an educational spin on them. Charmed Labs has been the device design house for his lab.

The CMU Cam goes back to [the first release] around 2000-2001, and the Pixy Cam is actually the CMU Cam number five because it’s gone through several revisions over the years. It’s now more capable and less expensive.

I became involved on a bit of a lark; I had found this processor in the tech news and I thought that would make a good camera chip processor for a low-cost camera so I sent it off to Illah and I guess at that particular time the CMU Cam  was in need of an update, and so he [asked if I wanted to work on the next version]. I didn’t really have anything in mind, but it sounded like fun. So that’s how I go into Pixy. It was a different project for me.

We had a crowdfunding campaign that launched in late 2013, and the crowdfunding was successful, so I’ve been making sure the project's been up to date. This spring is we came out with Pixy 2 and it’s followed the trend of becoming smaller, lower cost, and better performance. It’s been a fun project.

AAC: What has been the most interesting challenge while developing the Pixy cam?

RL: I think, for me, it’s always how do you find a way for it to be used by the largest amount of people? I found that the way you can get it in front of the most people is cost, which is a huge driver. In addition to cost, how easy it is to use. How much knowledge do you need to get his thing working with whatever thing you want to use it with?

Those two problems, cost and ease of use, from the engineering and product design perspective, can be really challenging. I would say definitely those two things are the most challenging parts of Pixy.

...Cost and ease of use, from the engineering and product design perspective, can be really challenging.

AAC: What have been some of the most interesting applications of the Pixy cam?

RL: There’s a gentleman in Georgia who started his own Kickstarter that uses Pixy as a way to aid in drone navigation. Microsoft used Pixy in one of their Windows 10 demonstrations where Pixy helps a computer play air hockey. There’s also Ben Heck who has his own YouTube channel and his show... did a couple of episodes that featured Pixy.

It’s been fun, all these projects are unique and they definitely have a fun factor.

AAC: What do you think has been the most interesting development in robotics over the past few years?

RL: There are so many different areas of robotics. I think I’ve always been in this robotic niche that I call educational robotics, using robots to teach science, engineering, and technology concepts. That’s kind of my focus. I think it’s fun to interact with that audience.

There’s a lot of overlap with the Maker movement. I would say that the most exciting thing for me is this explosion of the Maker movement, where people are being inspired to make stuff. The Maker movement is similar to robotics in that it just has so many facets—it’s so huge no one can really define its boundaries. I think it’s cool that a lot more people are paying attention to making things, and in particular, in my field, making cool robots.

AAC: Speaking of your specialty in educational robotics, what do you think is the biggest misconception when it comes to learning about robotics?

RL: I would say it’s really easy to underestimate the complexity of a robotics problem just because we, as humans, find the everyday tasks that robots do to be really simple. So a computer has no problem solving a 10th order differential equation, whereas that’s a really hard problem for a person to solve.

Folding a towel is a really hard problem for a robot and it’s really easy to fall into that trap where it seems like an easy problem. So when you try to design a robotic system to do that same task, it can be disappointing or discouraging and can be a lot harder than you thought it would be. As humans, we perceive these things that robots are doing or trying to do as really easy things to accomplish.

...It's really easy to underestimate the complexity of a robotics problem just because we, as humans, find the everyday tasks that robots do to be really simple... Folding a towel is a really hard problem for a robot.

AAC: Perhaps you could walk us through how a roboticist might break down the towel folding task piece by piece. Could you describe how you might approach that problem at a high level?

RL: The towel folding problem I should say has been solved by a couple of companies. Willow Garage made a robot that can do this a couple of years ago and it only took the robot about 30 minutes. But it’s just funny [because] when I watch the video [I think about how] someone spent a lot of time breaking down the problem of folding a towel.

 

Willow Garage's PR2 folding a towel in 2013. Screenshot used courtesy of Johan Voets.

 

The technical challenges are interesting [even if] the actual task of folding a towel isn’t that interesting. [However], what that person had to do was break down the problem into a bunch of really specific perceptual and actuator steps. 

First, you need to find the towel and then try to estimate its position and angle relative to the robots manipulators, and then you’ve got to manipulate the towel specifically, maybe finding a corner of the towel and picking it up. Even that first task, locating the corner of the towel and grasping it, is a challenging, graduate-level problem.

Then, the rest of the task. You’ve got to pick up another corner then you’ve got to pick up the towel with one gripper, hold both corners, then move the other gripper over. Then try to find the place where you can hold the towel so that you can pull it tight into a nice rectangle.

When you break down that really simple problem, it looks really complex when you break it down into perceptual and actuation tasks. It’s a really hard problem. 

AAC: What's something that inspires you most about the work that you do?

RL: One thing that’s cropped up for me lately: I think it’s an amazing time in engineering [based on the] fact that you can buy a $35 computer that just 20 years ago would cost several millions of dollars. Technology moves so fast, and as engineers sometimes it’s just fun to stop and look around and think wow, we’ve come a huge way. The [technological] world is a lot more interesting than it was a couple of years ago. 

Another quick story: I had the privilege to program a Cray-2 back int he 80s. I had a poster of a Cray-2 on my bedroom wall.

 

The Cray-2. Image courtesy of Cray Super Computers.

 

Most kids would have a poster of a Lamborghini. Well, I had a poster of a Cray-2. I remember [being told the Cray-2] could do so many billion floating point operations per second and I thought 'wow, that is amazing.'

[Recently], I looked into how a Raspberry Pi compares to a Cray-2. Cray-2 can do two billion floating point operations per second, which was pretty amazing back in the 80s. The Raspberry Pi can do six billion floating point operations per second, which is pretty remarkable. The Cray-2 costs $40 million and each Cray-2—there were only a handful—each one had hundreds of scientists working on it.

Today, we can buy a $35 Raspberry Pi and we might use it to control some LEDs. That’s what progress is. You’re not really concerned about wasting computing power, because it’s so available. 

 

AAC: Thank you for sharing your thoughts with us, Rich!