Neural networks are one of the machine learning models that mimic the functioning of the human brain and are an indispensable technology for artificial intelligence (AI). It can capture complex data relationships and is used in various fields such as image recognition, speech recognition, and natural language processing. It is said that artificial intelligence technology will become increasingly important in the future. To use it effectively, understanding the mechanism and role of neural networks is a plus. This article will introduce an overview of neural networks, methodology, types, and specific use cases.
What is a Neural Network?
A neural network is a machine learning model that mimics the functioning of the human brain and a mathematical model of artificial neurons (nodes) and synapses that represent neural networks in the brain. It consists of an input layer, a hidden layer (intermediate layer), and an output layer, and the neurons in each layer are connected to form a network. In the figure below, there is only one hidden layer. Still, a Deep Neural Network (DNN) is a multi-layered hidden layer that improves the expression and is used to improve the learning accuracy called learning.
Examples of the use of neural networks.
Neural networks can understand complex data relationships and are used in various fields such as image recognition, speech recognition, and natural language processing. For example, in handwritten character recognition using image recognition, the input image is analyzed pixel by pixel, and the characters are recognized based on the information contained in each pixel. Similarly, speech recognition identifies words and sentences from audio data. These technologies have been applied to smartphone voice assistants and automatic translation systems to improve daily life and work efficiency. They are also used in more advanced technology areas such as driverless vehicles and image diagnosis using medical images.
How do neural networks work?
How does a neural network learn from input data? Also, what is a deep neural network with more hidden layers? We will provide an overview of each.
Basic neural network (NN) mechanism
As explained in the beginning, a basic neural network (NN) consists of three types of layers: input layer, hidden layer, and output layer. The input layer is responsible for acquiring the data, the hidden layer extracts features from the data, and the output layer outputs the final prediction or classification results. The connections (synapses) between neurons in each layer have a “weight” that indicates how important the values of the neurons in the previous layer are to the next neuron. Adjustments are also made through the neural network’s own tuning parameter called “bias”, which is replaced by an “activation function” that determines how the output is sent to the next neuron.
Thus, the input layer, hidden layer, and output layer are connected like a relay to produce the final result. These calculations learn complex data patterns and make predictions and classifications. For example, in the case of handwritten character recognition, the pixel information of the input image data is transferred to the input layer, the features of the data are extracted to the hidden layer, and the characters are recognized in the output layer.
Neural networks learn by repeatedly inputting and outputting data and optimizing the weights and other parameters. You will be able to achieve better results by making fine-tunings to increase the accuracy of the model, such as how the weights are applied.
How Do Deep Neural Networks (DNNs) Work?
A deep neural network (DNN) is a neural network (NN) consisting of four or more layers, with the hidden layer being multiplied by an input layer, a hidden layer, and an output layer. Currently, this method is widely used, and some complex models have more than 100 hidden layers. For example, when determining the type of animal appearing in an input image, a shallow layer recognizes the color and shape of the animal in the image, and the next layer searches past data to find similar animals. The output layer then concludes which animals are included in the image data. Remember that machine learning is also performed when searching for data similar to past data, so the more training data is collected, the more neural networks can be applied, and the expected improvement in accuracy can be achieved.
Therefore, increasing the number of hidden layers makes it possible to process complex information, but it is not uncommon for more hidden layers to be more accurate, thus requiring appropriate model design.
How to train neural networks
The goal of learning in neural networks is to achieve the desired result at the output layer. There are various learning methods in machine learning that are suitable for the target. Here we will introduce the difference between general “unsupervised learning” and “supervised learning”.
Supervised learning
Supervised learning is a learning method that relies on data of correct answers prepared by humans in advance. Learning is performed using labeled data called training data or teacher data. For example, by importing a large number of images labeled “dog” or “cat”, the system learns the characteristics of dogs and cats. Then, when a new image is presented, it recognizes and predicts whether the object in that image is a dog or a cat based on the data learned so far. This supervised learning is used in areas such as demand forecasting, weather analysis, and speech recognition, where the results are predicted based on past data.
Unsupervised learning
In unsupervised learning, there is no labeled data for the correct answer and the correct answer is obtained using the input data. Since we analyze the characteristics of the input data, we mainly perform tasks such as discovering common patterns, classifying them, and simplifying the data. Common examples include “clustering” and “dimensional reduction”. Clustering is the process of grouping data according to its features. Dimensionality reduction, as explained in the Autoencoders section, can reduce the number of neurons (dimensions) and extract only the necessary information that determines the characteristics of the data. Unsupervised learning is used in areas such as image recognition in autonomous driving technology and anomaly detection in visual inspection.
Difference Between Machine Learning and Deep Learning
Machine learning is a technology that allows artificial intelligence to learn autonomously rather than relying on explicit instructions (programs) from humans. It involves working repeatedly with large amounts of data, finding patterns and regularities, and learning in a way that can be used for analysis. Deep learning is a further development of machine learning and is based on the currently introduced neural networks and uses processes similar to human recognition processes to obtain answers.
For example, when a machine recognizes a dog, it makes a decision based on the target’s features in the image, such as the shape and outline of the ears, and compares this with past data. However, a human reaches a conclusion through a classification process, such as recognizing the whole and examining the details, seeing the details, and understanding the whole. Similarly, deep learning analyzes data in multiple layers. Additionally, AI improves accuracy by learning where to focus and find features.
For example, in typical machine learning, humans are intentionally instructed to “compare colors,” but with deep learning, the machine also learns where features appear in the data and also learns how to recognize colors. This allows for more autonomous learning.
Types of Neural Networks
In addition to the Deep Neural Network (DNN) described above, there are several types of neural networks. Here we introduce the characteristics and mechanisms of specific neural networks.
Convolutional Neural Network (CNN)
A convolutional neural network (CNN) is a type of deep neural network that is particularly good at processing information from images and videos. It is mostly good at “general object recognition”, having special layers such as “convolution layers” and “pool layers” that categorize images and allow you to automatically localize images.
In the case of a normal neural network, synapses are fully connected to all neurons (fully connected), but in the case of a convolutional neural network, the connections between neurons are limited to the convolution layer, and edges are created using attributes. Convolution operations. In addition, a “pool layer” extracts and compresses only the particularly important information, removing noise and emphasizing features. Then, operations such as classification and prediction are performed using a traditional neural network-like structure called a “fully connected layer”.
Recurrent Neural Network (RNN)
A recurrent neural network (RNN) is a neural network that is good at processing time series data, which has attracted attention in areas such as natural language processing. For example, if you want to predict the next word in a sentence, you need to remember what the previous word was. This is made possible by a function called “recalling previously calculated information”.
Instead of multiple hidden layers, it consists of three layers: the input layer, the hidden layer, and the output layer, and the calculation results of the hidden layer are output to the output layer, while the calculation result is returned to the hidden layer. Recounting By repeatedly (re)assigning the output from the hidden layer to the hidden layer, it is possible to clarify the concept of “how many assignments have been made now”, since the same hidden layer is used multiple times. Added “axis (state axis). This allows you to learn patterns in continuous data like text strings and time series data like stock quotes.
Generative Adversarial Network (GAN)
A generative adversarial network (GAN) is a model that learns and generates fake data based on generated data. There are two different types: a “generator” that generates fake data, and a “discriminator” that determines whether the input data is real data or fake data generated by the generator. The generator generates more complex data to fool the classifier, and the classifiers compete to look at the data generated by the generator, increasing the accuracy of their identification. This competition allows the generators to be more precise and generate data that is as realistic as possible.
When generating a new image using a GAN, the generator initially draws it randomly, but the classifier quickly determines that it is fake. The generator takes this feedback and tries to draw a more realistic image. By repeating this process, you will be able to draw a more realistic image. This generative adversarial network is applicable not only to image generation but also to a wide range of fields such as speech synthesis and text generation.
Self-Encoder (Auto-Encoder)
Auto-encoder is a model that encodes and compresses the input information (encoding), transforms it into something else, and, then returns it to its original form (decoding) and outputs it. Why do we need this technique, which at first glance seems like a waste of time since its input and output are exactly the same?
As mentioned above, neural networks allow for complex operations by stacking layers. However, in reality, there is a problem called “vanishing gradient” where accuracy decreases when many layers are combined. There is also the problem of “overfitting”, which means a loss of efficiency due to overfitting of the training data. Auto-encoders overcome these difficulties.
The figure below shows a simplified version of the auto-encoder. In this example, input “3” and output “3”. At this point, we want to focus on the number of neurons in the hidden layer (yellow circle). Since there are fewer neurons than the input and output layers, the information must be reduced (encoded/compressed) in some way. In order to obtain the same output “3” despite the information reduction, necessary features such as “Which information can be skipped without any problems?” and “Which information can be skipped?” need to be determined. Is the information important?” An autoencoder is a system that applies this method to each layer of a multilayer neural network to significantly reduce errors and improve learning performance. This method can also be applied to “anomaly detection” to determine whether the input data is normal or abnormal, and is used, for example, when performing visual inspection.
Specific uses of neural networks
Neural networks are used in a wide range of fields, including medical image evaluation, marketing through behavioral analysis, financial forecasting, and energy demand forecasting. Here we will introduce three particularly important applications.
1: Computer Vision
Computer vision is a technology that allows us to extract and understand important information from images and videos. Using image recognition using neural networks, it is possible to analyze image and video data and identify people and objects. A familiar example is facial recognition, which uses a smartphone camera to recognize a person’s face and authenticate their login. Cameras installed in cars are also used for driving support functions such as detecting other vehicles and pedestrians and recognizing lane markings. In addition, in the medical field, they support image diagnosis by analyzing medical images such as CT and MRI images, helping to miss diseases and reducing the time required for image interpretation.
2: Natural Language Processing (NLP)
Natural Language Processing (NLP) is a technology that understands the context of human language and generates sentences. Neural networks also play an important role in this field and are used for automatic translation of foreign languages and summarization of news. Chatbots, which are also increasingly adopted in customer support for products and services, are used to correctly understand the meaning of free texts entered by customers, thus creating an A system that allows them to give the right answers. In addition, in the field of social media analytics and market research, the use of large amounts of text data is increasing to analyze the sentiment from published content such as reviews and to understand user opinions and trends.
3: Recommendation engine
Neural networks are also used in recommendation engines that recommend products and content (items) to users based on certain rules and behavioral data. For example, we analyze data on users’ past behavior (clicks, pages viewed, purchase history, etc.) and use this data to understand the elements we are considering and analyze the similarities between them, as well as to predict the behavior and each element. This makes it possible to combine the results of user behavior analysis with the matching of the element that interests the user. Maso.
Summary
Neural networks are a type of machine learning model that mimics the functioning of the human brain and are used in various fields such as image recognition, speech recognition, and natural language processing. By learning from complex data, neural networks can be used for prediction and classification, and can be used to increase operational efficiency, reduce costs, and enhance competitiveness in various sectors such as medicine, finance, and manufacturing. Neural networks are expected to continue to play a central role in technological innovation and contribute to the development of the information society.