3 years ago



Use deep learning to

Use deep learning to simplify and accelerate machine vision implementation AUTOMATION Deep learning is now not only a method reserved for researchers with highly specialised skills. Nowadays many tools are free, hardware costs are low and training data is available free of charge. This also opens up completely new possibilities for industrial image processing, for example in the detection of anomalies. The article explains in 5 steps the building and deploying of neural networks for deep learning. The saying “a picture is worth a thousand words” never rang so true in the world of machine vision as it does today. With deep learning, thousands, even millions of lines of code can be replaced by a simple neural network trained with images and very little coding. The great news is that deep learning is no longer a method available only to researchers or people with highly specialized skills and/or big budgets. Today, many tools are free, tutorials are easy to find, hardware cost is low and even training data is available at no cost. This presents both opportunities and threats - as new players emerge to disrupt established names and spur innovation. It also provides opportunities for your machine vision systems to do things previously unimaginable – as an example, deep learning can be used to recognize unexpected anomalies, typically very difficult or almost impossible to achieve with traditional coding. A noteworthy benefit of deploying deep learning neural networks is that it allows complex decisions to be performed on the edge with minimal hardware and very little processing power – Low cost ARM or FPGA based systems and new inference cameras like the Flir Firefly DL make this possible. What is deep learning: the fundamentals Deep learning is a subset of machine learning inspired by how the human brain works. The thing that makes deep learning “deep” is the fact that there are multiple “layers” of neurons of various weights which help a neural network make its decision. Deep learning can be broken into two stages, training and inference. During the training phase, you define the number of neurons and layers your neural network will be comprised of and expose it to labeled training data. With this data, the neural network learns on its own what is ‘good’ or ‘bad’. For example, if you are grading fruits, you would show the neural network images of fruits labeled “Grade A”, “Grade B”, “Grade C”, and so on. The neural network then figures out properties of each grade; such as size, shape, color, consistency of color and so on. You don’t need to manually define these characteristics or even program what is too big or too small, the neural network trains itself. Once the training stage is over, the outcome is a trained neural network. The process of evaluating new images using a neural network to make decisions on is called inference. When you present the trained neural network with a new image, it will provide an inference (i.e. an answer): such as “Grade A with 95% confidence.” STEP 1: developing a deep learning application The development of a deep learning application broadly entails 5 steps. A high-level description of each step follows. Step 1: Identify 12 WORLD OF INDUSTRIES 2020

01 Classification used to identify flawed solders using a FLIR Firefly DL it can be utilized to detect a person breaching a safety parameter around robots on a production line or identify a single bad part on a production/assembly line conveyor system. SEGMENTATION: The third type of deep learning is segmentation; typically used to identify which pixels in an image belong to which corresponding objects. Segmentation is ideal for applications where determining the context of an object and it’s relationship to each other are required (for example autonomous vehicle / Advanced Driver Assistance Systems, popularly referred to as ADAS). ANOMALY DETECTION: This type of deep learning task can be utilized to identify regions that do not match a pattern. A typical example of an application where anomaly detection can add value would be stock control and inventory management at grocery stores. This application involves using an inference camera to detect and highlight shelves that are empty or about to need replenishment, providing real-time notifications and improving efficiency. PASS In this article, you will learn: n Basic glossary used in deep learning n Types of machine vision tasks deep learning is most suitable for n The 5 steps for developing and deploying a neural network for inference on the edge FAIL the appropriate deep learning function: In the world of deep learning, tasks are classified into several functions. The ones we consider most common to machine vision are: CLASSIFICATION: First and most basic application of deep learning is classification. The process involves sorting images into different classes and grouping images based on common properties. As an example, you can use classification to separate a flawed part from a good one on a production line for quality control or while conducting PCB solder inspections – like illustrated in Fig.1 (using a Flir Firefly DL inference camera). DETECTION AND LOCALIZATION: Another deep learning task ideal for machine vision is called detection and localization. Using this function, you can identify features in an image and provide bounding box coordinates to determine its position and size. For example, n Available tools and frameworks to get started n Tips on making the process easier n Potential shortcomings of deep learning to consider STEP 2: key advantages and disadvantages of available frameworks Once you determine the deep learning function you intend to use, you’ll need a toolset (developers call this a “framework”) best suited to your needs. These frameworks will provide a choice of starter neural networks and tools for training and testing the network. With some of the world’s biggest technology companies vying for dominant positions in the deep learning market, frameworks like TensorFlow by Google, Caffe2 by Facebook and OpenVino by Intel (all free) demonstrate the quantum of investments and resources flowing into the deep learning market. On the other end of that spectrum, you also have Pytorch, an open source solution thats now part of Facebook. These tools are easy to use and provide great documentation (including examples), so even a novice user can train and deploy a neural network with minimum effort. Discussing all available frameworks would warrant a separate article, but the following pointers list key advantages and disadvantages for 3 of the most popular frameworks: PYTORCH: simple and easy to use, used in many research projects, not commonly used for large deployment and only fully supported for Python TENSORFLOW: large user base with good documentation, higher learning curve compared to Pytorch, offers scalable production deployment and supports mobile deployment CAFFE2: lightweight, translating to efficient deployment, one of the oldest frameworks (widely supported libraries for CNNs and computer vision), best suited for mobile devices using OpenCV. The neural network you choose would eventually depend on the complexity of the task at hand and how fast your inference needs to run. Furthermore, even companies lacking the resources and employees to learn and implement a deep learning solution can rely on third party consultants. One such example is Enigma Pattern ( STEP 3: preparing training data for the neural network Depending on the type of data you want to evaluate, you’ll require a repository of images with as many characteristics you hope to utilize in your evaluation and they need to be labelled appropriately As an example, if your neural network needs to identify a good solder from a bad one, it would need hundreds of variations of what a WORLD OF INDUSTRIES 2020 13