Franziska Kirstein is a human-robot interaction expert at Blue Ocean Robotics ApS in Odense, Denmark. Her responsibilities are execution of EU-funded projects, in particular projects with relation to Human-Robot Interaction. She's also involved in development projects in user studies, user needs analysis and user testing.
Franziska Kirstein talks about her experience as a linguist working with human robot interaction.
We get to hear about what works and what doesn’t when non-engineer users are tasked with teaching robots different movements.
Franziska also describes some of the challenges with kinestetic guidance and alternative methods that can be used.
She then talks about some of the projects she is involved in, including one in robot assisted health care and one involving social robots.
This podcast is part of the Wevolver network. Wevolver is a platform & community providing engineers informative content to help them innovate.
Learn more at Wevolver.com
If you are interested in sponsoring the podcast, you can contact us at richard@wevolver.com
Per Sjoborg: Welcome to the podcast version of Robots in Depth. This episode’s interview is with Franziska Kirstein and is produced in cooperation with Wevolver. Robots in Depth is supported by Aptomica. Visit Aptomica.com to connect. You will find all past episodes and more in RobotsInDepth.com. Welcome to this episode of Robots in Depth. Today I’m here with Franziska Kirstein and she's from Blue Ocean Robotics and we're going to talk about many things but we're going to start at the beginning like I usually do. When did you discover the field of robotics? How come you're in the field right now?
Franziska: I studied language and communication first so I'm not an engineer as a background. Then in my master my professor Castine Fisher, she is also a linguist with background but she's working in human robot interaction so she kind of introduced me to the field. I wrote my master thesis with her and that's how I got into the field.
Per: What was your thesis?
Franziska: It was about different control modalities for programming by demonstration.
Per: This is where we show the robot what we want to do rather than telling it exactly. Could you give us a brief overview of that work?
Franziska: So we invited normal students and users, university staff from the library or administration. People that didn't have any knowledge about robotics. They might have a technical background some of the students did but they didn't know anything about robotics and we invited them. All in all the whole study that we did was around 200 people that we invited. For the first study, for the three modalities that we tested we invited 50 participants and we asked them to try and control the robot because we wanted to find out the issues with those modalities. The modalities were the universal robot control panel. It’s a bigger one where you control the robot with by pushing buttons and you're usually controlling the robot only one joint at the time. That means that you can only make one movement with the robot and then of course kinesthetic guidance. That’s a widely used method where you just take the robot arm and move it and show the robot the movements that you want it to learn. There you can actually move all joints at one time, very flexible. Then we had a control glove. That is a glove and that has a sensor here on top and with this sensor you're controlling a point in the robot and that's how you control the robot and can teach to robot.
Per: What was the outcome? Which method was to prefer? Were there different method for different situations were?
Franziska: That was definitely kinesthetic guidance that was the best in all categories. We tested the length and the accuracy of the demonstrated trajectories that is if the demonstration was like that or if it was like that.
Per: How complicated the move was.
Franziska: If it's like that the robot, it cannot really easily learn the movement but if it's a straight movement and it's easier for the robot to learn.
Per: Basically how noisy...
Franziska: Yes.
Per: Basically how much noise did the human introduce?
Franziska: That was the point, how can you make that better and then we also counted the arrows that the participants made like bringing the robot into a situation that is not good for the robot. Singularity when the joints are all aligned, when the robot is self-colliding or the gripper is pushed too much into the foam of the platform. That’s not a good situation for the robot. Engineers often see that but normal users they don't see that.
Per: They don't think about what a singularity is.
Franziska: They got to know that before but of course it was a lot to remember. An introduction video for five minutes and yes so they often brought the robot into a situation that was not that good. Our engineer was a bit nervous because it was the first time such a study was done at that institute because it's just not normal to have normal people for something like that.
Per: That anyone come close to your robot.
Franziska: Your very expensive robot.
Per: That makes you nervous. I can understand that. That’s interesting result so the reasonably common kinesthetic guidance was still the best. Are you looking into how we can improve that part because of course it has its challenges too?
Franziska: Kinesthetic guidance, it has its challenges definitely. We used a universal robot arm which is quite small but still it was a platform with a big table two universal robot arms. We only used one but in order for to position the robot they needed to go the right side and then on the left side and choose between the situations and then they needed to grab on the other side of the robot which makes it very uncomfortable. After a while the gripper also got very hot so when they wanted to rotate it then that was not that nice and of course kinesthetic guidance is very good but when you have a large robot, universal robot arm is not that large now but if you have a large robot you won't be able to do kinesthetic guidance. Also in factories where there's not that much space you won't be able to go around the table and teach the robot. You want to use a tiller operated, tiller operation device.
Per: Are you then continuing development of this pendant, this new method you developed?
Franziska: Yes. We wanted to take all the advantages from kinesthetic guidance and put them into a tiller operation device. We did that because we noticed that participants were switching control points a lot of times so they were touching the robot here and then they went to the other side and touched it there. We thought that developing a device that makes use of this switching control points could be a good solution. We made a device where you can actually intuitively switch between control points and in addition those control points control different points in the robot. Actually I have I have the bottle here.
Per: That is how it started I understand.
Franziska: This is how it started yes at the brainstorming session. We had this bottle and we thought this this is actually a good solution. It looks like the robot arm like the gripper and if you have it on top you can do the large movement which is very stable and you're doing those movements with this control point but if you want to do fine tuning, if you want to insert a piece into something in an assembly like a normal peg and hole.
Per: Which is most in assembly. We’re talking about peg and hole but putting a shell on a cell phone and putting a bolt in a hole all of these are peg in a hole things.
Franziska: We did it with a Cranfield set but I mean as you say it can be applied and in any assembly situations.
Per: Anything where two pieces with a snug fit has to go together.
Franziska: There you need fine-tuning all the time so this resembles kind of like a pen. Something that is very good for fine-tuning and this is how the device more or less looks like with a bit more nicer design. With this this point down here you control a point in the robot that is at the tips of the gripper so at the fingertips of the gripper actually, the lowest point that you can probably get. That's also where users focus on when they insert a piece because you're looking at the piece, you're looking at the hole where you want to insert the piece and doing that yes and it feels much more stable when you control an upper point in the robot.
Per: You don't have to reach around the robot. I guess since you're doing it fly-by-wire so to speak or fly by Wi-Fi in this thing I would assume. You can't manipulate the robot in such a way that it will come into undesirable positions and hurt neither you or itself.
Franziska: Not me or anyone else. I mean you could you could still put it in a position that is not good for the robot.
Per: That is a lesser problem. Not good for people is worst.
Franziska: No but not good for people that's because you can be far away and depending on which sensor now I'm not a technical person but that's what our engineers told us depending on the sensor that you put in you can be five meters away or twenty or maybe in another room.
Per: In another room is interesting because that means you can't monitor robots from a larger field and also when you're further away from a larger robot it's safer but it's also so that I've used the Da Vinci surgical robot. You could probably control a very small robot that you physically couldn't touch because it scales.
Franziska: Yes that's true. I'm not sure we haven't thought about how you could use the device in other situations but basically it controls a robot arm so any other situation where a robot arm is used you could use the device.
Per: Is the device work in progress or you continue to work on it?
Franziska: It's work in progress so the next step is actually because we haven't involved any real end users. We have involved users that didn't know anything about robots or how to control robots but it's also very necessary to include the end users, those who use it maybe in a production line or somewhere to program a robot. That would be very nice to have a workshop with them and then make some redesigns based on their suggestions.
Per: What do you do at Blue Ocean Robotics?
Franziska: At Blue Ocean Robotics I am human robot interaction expert because I have focused so much on HRI before and that is not only focused on industrial robots. I am involved in development of healthcare robots. We have the new UV disinfection robot that I think it's since one and a half two years we've been developing that and the first prototype is ready now. What I usually do there in our development projects is analyzing the end-user situation. What do they need? What is their normal work procedure? How can the robot fit in there in the future? Are they scared of the robot? Would they accept the robot? How can we make them accept the robot? Those are very interesting questions that as you say the end user needs to be involved right in the beginning as early as possible and that's usually what we do. Then I kind of hand this over to our engineers and we have workshops and discussions and see okay, what is the most important part of usually, you'll get a dream list from the end-user. You’re doing robotics, can you solve this and this and this. That is of course very difficult so you really need to find the essential part that makes them happy in the end.
Per: Also the one that is reasonable to achieve robotically and financially.
Franziska: You don't want to promise them something and then in the end it doesn't work even though you might have technically solved the problem but it's just way too complicated.
Per: Or too expensive.
Franziska: Yes exactly or that.
Per: You're also involved with the Echord Plus Plus which is an EU funded the project focusing on research then.
Franziska: No actually blue ocean robotics is part of Echord Plus Plus because we always take research solutions that are ready for the market or maybe not ready yet. We take those and focus on how can we get them to the market so bridging research and market and that's why we're a partner in the project. Echord Plus Plus has a lot of different projects where they support experiments, research projects, ideas that have the potential to go to the market. We, the Blue Ocean Robotics is involved in the PDTI healthcare challenge that is public and user driven design where we have a public body involved that is a public body the hospital in Barcelona. There we develop together with them or actually we have chosen three teams that develop healthcare robot together with the hospital in Barcelona. That robot is for a geriatric assessment so the situation is that we have a doctor and that doctor every six months he's seeing the elderly or he or she is seeing the elderly person and has to assess how that person is doing cognitive, physically walking, if they need help, if there's anything in their daily life that they need help with. The doctor usually has 42 minutes to an hour to do that assessment and there are specific tests that the doctor needs to go through but there's not, I mean an hour maximum is not a lot of time to do the tests, do the evaluation, record the results so the robot is supposed to help that doctor to do that.
Per: That is very interesting.
Franziska: Very challenging also.
Per: But it's healthcare and assisted living is very strongly motivated because it can make people's life so much better if we succeed and when we succeed.
Franziska: For the doctor in that situation it can help the doctor to actually focus on the elderly person instead of focusing on writing something down or trying to remember everything that the doctor is seeing.
Per: You are doing more than just this assistive robots for doctors. Can you mention a few of the other projects within Blue Ocean that you're working on?
Franziska: I think the innovation projects that we do at Blue Ocean Robotics because that is something else I really like. The innovation project is mostly with municipalities together here in Odense or together with municipalities in Denmark, whole Denmark actually. We have a partnership with those municipalities. They buy robots from us. It can be any robot. It can be healthcare robot, transportation robots or beam, telepresence robots or social robots. My focus is the social robots so I'm most often in schools with teachers there. They buy a social robot whether it's a SinO or cueball or the Nao robot depending on what we think they need that's what we what we discuss with them before. Then we take care of that they can use the robots on a long term because it's their robot.
They have bought the robot and it's not only that we're doing a research study and we come into the school six weeks and then we're out again. We are there for sixteen weeks. We have a workshop in in the beginning where we teach them how to use it. More of the background of robotics a bit a bit, where it comes from and what those robots can, the social robots, what are the results from research, scientific results that is often very good for them to get an idea of how to use the robot. Then we have weekly contact while they are using the robots. In the middle we have a meeting again and then a final meeting to discuss also the problems. How we will move out so to say now. How will you move on and use this robot after we are gone so to say. I think that is it's very interesting because it's something so different from research that I've been doing before. It’s actually not making the solution ready for the market but actually using the solution in the market. Even though it's maybe not ready yet but it's getting ready because of the feedback from all the teachers.
Per: Do you see how the kids are interacting with the robots? That would be interesting to know more about.
Franziska: Sometimes and so I've been doing two projects so far and in one of the projects I will soon be able to beam in with the telepresence robot and see a session. We’ve seen lots of videos but also the children that they work with are always most often also children with specific needs or children with autism and then it's kind of distracting when there's a new person in there. There’s already a new robot in there, in the classroom and that's not so good but the project that we have right now it has been going on since Christmas and it's alright to bring another robot in so that I can see it. That’s going to be very interesting. I'm looking forward to that. The social robots that we have they are often for children with specific needs. We also have education robots but that is not me who is doing that.
Per: Could you mention some other projects you are also working on?
Franziska: We also have the Reconcile Project. It's a new European project where we are developing a reconfigurable work cell for small productions or SMEs that have a small production and need to be very adaptive in their production process so that they don't need to buy several machines because that is one of the problems that we have three partners actually, industry partners focusing or producing different things and one of them, he's producing parts for automobile industry so Audi, BMW. He said that they produce the parts and then I need to keep the machine for five to ten years because in order to be able to produce spare parts and that takes a lot of space for them. They don't need the machine they just need to keep it somewhere and that is very expensive for them to not be able to use the space so this would be something where you could easily reconfigure the work cell and you already actually don't need to have three machines there but you can adapt the production process and then have the part produce that they need, the spare part.
Per: Very interesting because logistics around spare parts and all that is of course it's a very big things so anything that could be done to make it easier and make more efficient is very significant. We are hearing lots about many interesting projects here. Could you share even more?
Franziska: I have one more project. I'm doing lots of lots of thing. I hope you liked this episode of the podcast version of Robots in Depth. This episode is produced together with Wevolver. Wevolver is a platform and community providing engineers informative content to help them innovate. It is how engineers stay cutting edge. Aptomica is the founding sponsor for Robots in Depth. Aptomica runs anything in modular robotics. Dream, rent, build. Visit Aptomica.com to connect. I am your host Per Sjöbor. Until the next episode thank you for listening.
END OF TRANSCRIPT