Workshop 4.0 – empowering robotic arms with computer vision
Robots can lift much more than humans, mostly because they won’t hurt their spines carrying heavy loads. If people get injured – they suffer and need treatment. Fixing a broken robot is usually a matter of replacing some gears and there is simply no suffering involved. Making robots lift heavy wooden panels makes people’s lives better, safer and more comfortable – a perfect spot for Tooploox to contribute.
According to WebTribunal, up to 400,000 new robots appear on the market every year with South Korea leading the way, already having 900 robots to every 10,000 employees.
Yet implementing a robot in the workplace is far more complicated than just buying one. The devil is in the details and this time it is all about the ability to grab a wooden panel from an unsorted pile.
The client
Workshop 4.0 is a startup focused on providing industrial robots combined with sensors and data processors to make the fourth industrial revolution a reality.
“We have chosen to engage in robotics because it allows us to automate repetitive and physically demanding tasks for the employees of TZ. Also because it allows us to improve productivity and remain competitive.”
This time, the company was to deliver a robotic arm to grab wooden panels from unsorted piles and precisely insert them into a CNC machine that would transform the raw wood into work-of-art quality doors. These are produced by cutting and milling the wooden panels into the desired shapes and applying – sometimes astonishingly sophisticated – patterns.
Grabbing any given thing from an unsorted pile is a no-brainer for humans. But for a robot? Far from easy.
The challenge
In this particular project there were several challenges to deal with:
-
Various sizes of wooden panels to be worked with
The workflow starts with the employee who comes into the workshop with a forklift and a pile of panels to be worked. The panels are put down in the form of a stable pile, with various angles, different sizes, and usually no predictable order. One of the reasons behind this is the business model of the company – it is craftsmanship, not mass production where scale follows standardization.
The workshop produces doors from the size of garage doors to kitchen cupboards – and basically everything in between.
-
Limited budget
The company assigned a budget for the task and fitting development into it was one of the primary challenges. This included the price of the system components – initial designs aimed to use depth estimation cameras, yet these are costly devices. Thus, building a system on budget and solving the challenges algorithmically was the only way to overcome this challenge.
-
Limited time
Not the least of challenges, the client expected the team to make the robot usable as soon as possible – this investment has to work for itself, and having the cash frozen in machines was counterproductive.
The end goal was to be realized in two and a half months.
-
The Distance from Poland to Switzerland
Implementation required the Tooploox team to operate at times in Switzerland. Also, the cooperation required the teams to synchronize using digital means and find the overlapping times that included the timezone shift.
-
Planning changes
During the project there was a significant shift from the initial planning.
“The bottleneck of the project was the displacement of integration timing by the client (extended beyond 6 months). For us, it was a real challenge to adapt our business planning and to ensure that the project partners remained committed for a longer period. This showed us that we had chosen the right partners (Tooploox) who proved resilient in supporting us throughout this integration, which took much longer than expected.”
Our work
The key challenge with an unstructured pile of panels was the fact that there was no easy way to show the robot the angles and edges of a particular piece of wood. Even the human eye could get confused with the complex wooden grain patterns and the limited variety of colors.
In the initial state, it was thought we could simply use a depth estimation camera, yet this is a costly device far exceeding the required budget.
Thus, the challenge had to be tackled by overcoming hardware limitations with superior coding.
The light web
The Tooploox team came up with the idea of using a laser CNC web projected on the pile from above. The web itself is regular, with clearly defined angles and lengths. Thus, spotting irregularities in the laser web pattern became our way to count the size and position of every panel in the pile.
-
The next step was to devise an algorithm to find cracks in the web and give information to the robotic arm on how to grab a particular panel and how to position it in the CNC milling machine.
-
The setup that made this approach possible was composed of two cameras that observed the pile from different angles and the laser projector installed directly above the pile.
-
The final component of the system was the connector between the vision system and the robotic arm, which delivers exact information on the position of the topmost panel and its size.
Calibration
The system had to be precisely calibrated, with a precisely defined spot where the pile of wooden panels had to be. The assignment of cameras was also crucial, with the algorithm being vulnerable even to slight changes of observation angle.
The effect
The Tooploox team was able to deliver a system that enables the robotic arm to precisely grab any wooden panel and place it with a precision of centimeters into the CNC milling machine.
The team delivered a fully-functional set of devices and software while fitting into the budget and delivered on time.
“We chose Tooploox on the recommendation of EPFL Lausanne. Tooploox’s approach was very open, accessible, and based on our client’s needs. We quickly established a bond of trust and were reassured by this remote collaboration between Switzerland and Poland. Edyta’s responsiveness and Krzysztof’s experience were very much appreciated.”
Let our specialists solve the problems and tackle the challenges that hold you from conquering the world.
Let’s talk