Workshop 4.0 – empowering robotic arms with computer vision

Workshop 4.0 – empowering robotic arms with computer vision

Robots can lift much more than humans, mostly because they won’t hurt their spines carrying heavy loads. If people get injured – they suffer and need treatment. Fixing a broken robot is usually a matter of replacing some gears and there is simply no suffering involved. Making robots lift heavy wooden panels makes people’s lives better, safer and more comfortable – a perfect spot for Tooploox to contribute.

Workshop 4.0 – empowering robotic arms with computer vision

According to WebTribunal, up to 400,000 new robots appear on the market every year with South Korea leading the way, already having 900 robots to every 10,000 employees. 

Yet implementing a robot in the workplace is far more complicated than just buying one. The devil is in the details and this time it is all about the ability to grab a wooden panel from an unsorted pile.

The client

Workshop 4.0 is a startup focused on providing industrial robots combined with sensors and data processors to make the fourth industrial revolution a reality. 

“We have chosen to engage in robotics because it allows us to automate repetitive and physically demanding tasks for the employees of TZ. Also because it allows us to improve productivity and remain competitive.”

This time, the company was to deliver a robotic arm to grab wooden panels from unsorted piles and precisely insert them into a CNC machine that would transform the raw wood into work-of-art quality doors. These are produced by cutting and milling the wooden panels into the desired shapes and applying – sometimes astonishingly sophisticated – patterns. 

Grabbing any given thing from an unsorted pile is a no-brainer for humans. But for a robot? Far from easy.

The challenge

In this particular project there were several challenges to deal with:

  • Icon check

    Various sizes of wooden panels to be worked with

    The workflow starts with the employee who comes into the workshop with a forklift and a pile of panels to be worked. The panels are put down in the form of a stable pile, with various angles, different sizes, and usually no predictable order. One of the reasons behind this is the business model of the company – it is craftsmanship, not mass production where scale follows standardization. 

    The workshop produces doors from the size of garage doors to kitchen cupboards  – and basically everything in between.

  • Icon check

    Limited budget

    The company assigned a budget for the task and fitting development into it was one of the primary challenges. This included the price of the system components – initial designs aimed to use depth estimation cameras, yet these are costly devices. Thus, building a system on budget and solving the challenges algorithmically was the only way to overcome this challenge.

  • Icon check

    Limited time

    Not the least of challenges, the client expected the team to make the robot usable as soon as possible – this investment has to work for itself, and having the cash frozen in machines was counterproductive. 

    The end goal was to be realized in two and a half months.

  • Icon check

    The Distance from Poland to Switzerland

    Implementation required the Tooploox team to operate at times in Switzerland. Also, the cooperation required the teams to synchronize using digital means and find the overlapping times that included the timezone shift.

  • Icon check

    Planning changes

    During the project there was a significant shift from the initial planning. 

    “The bottleneck of the project was the displacement of integration timing by the client (extended beyond 6 months). For us, it was a real challenge to adapt our business planning and to ensure that the project partners remained committed for a longer period. This showed us that we had chosen the right partners (Tooploox) who proved resilient in supporting us throughout this integration, which took much longer than expected.”

Our work

The key challenge with an unstructured pile of panels was the fact that there was no easy way to show the robot the angles and edges of a particular piece of wood. Even the human eye could get confused with the complex wooden grain patterns and the limited variety of colors. 

In the initial state, it was thought we could simply use a depth estimation camera, yet this is a costly device far exceeding the required budget. 

Thus, the challenge had to be tackled by overcoming hardware limitations with superior coding.

Icon play

The light web

The Tooploox team came up with the idea of using a laser CNC web projected on the pile from above. The web itself is regular, with clearly defined angles and lengths. Thus, spotting irregularities in the laser web pattern became our way to count the size and position of every panel in the pile. 

  • The next step was to devise an algorithm to find cracks in the web and give information to the robotic arm on how to grab a particular panel and how to position it in the CNC milling machine.

  • The setup that made this approach possible was composed of two cameras that observed the pile from different angles and the laser projector installed directly above the pile.

  • The final component of the system was the connector between the vision system and the robotic arm, which delivers exact information on the position of the topmost panel and its size.

Calibration

The system had to be precisely calibrated, with a precisely defined spot where the pile of wooden panels had to be. The assignment of cameras was also crucial, with the algorithm being vulnerable even to slight changes of observation angle.

The effect

The Tooploox team was able to deliver a system that enables the robotic arm to precisely grab any wooden panel and place it with a precision of centimeters into the CNC milling machine. 

The team delivered a fully-functional set of devices and software while fitting into the budget and delivered on time.

Workshop 4.0 – empowering robotic arms with computer vision - The effect

“We chose Tooploox on the recommendation of EPFL Lausanne. Tooploox’s approach was very open, accessible, and based on our client’s needs. We quickly established a bond of trust and were reassured by this remote collaboration between Switzerland and Poland. Edyta’s responsiveness and Krzysztof’s experience were very much appreciated.”

Decore Serpent

Let our specialists solve the problems and tackle the challenges that hold you from conquering the world.

Let’s talk