Development of a neural network with forward connection to classify Nepali writing patterns (OCR)

1. Objective

Design and program neural networks based on the Multilayer Perceptron as a pattern classification tool.

2. Theoretical part

Neural Networks

• Definition

Neural networks are massively parallel interconnected networks of simple (usually adaptive) elements and with hierarchical organization, which try to interact with real-world objects in the same way that the biological nervous system does.

•  Advantage

-Adaptive Learning

Ability to learn, to perform tasks based on training or initial experience

-Self-organization

A neural network can create its own organization or representation of the information that it receives through a learning stage.

-Fault tolerance

The partial destruction of a network leads to a degradation of its structure; however, some network capabilities can be retained, even suffering great damage

-Real-time operation

Neural computations can be performed in parallel; For this, machines with special hardware are designed and manufactured to obtain this capacity.

-Easy insertion into existing technology

You can obtain specialized chips for neural networks that improve their capacity in certain tasks. This will facilitate modular integration into existing systems.

• Basic elements of a neural network

Example of a fully connected neural network.

It is made up of neurons interconnected and arranged in three layers (the latter may vary). Data enters through the "input layer", passes through the "hidden layer" and exits through the "output layer". It is worth mentioning that the hidden layer can be made up of several layers.

• Neural network applications

Neural networks can be used in a large number and variety of applications, both commercial and military:

-Biology: Learn more about the brain and other systems, obtain retinal models.

-Company: Evaluation of the probability of geological and oil formations, identification of candidates for specific positions, exploitation of databases, etc.

-Environment: Analyze trends and patterns or weather forecast (climate).

-Medicine: - Speech analyzers to aid in hearing for deaf people, diagnosis and treatment based on symptoms and / or analytical data, monitoring in surgeries, etc.

Most of these applications consist of performing a pattern recognition, such as: looking for a pattern in a series of examples, classifying patterns, completing a signal from partial values, or reconstructing the correct pattern from a distorted one

Keras 

Keras is a high-level API from TensorFlow to build and train deep learning models. It is used for rapid prototyping, state-of-the-art research, and production, with three key benefits:

• User friendly

Keras has a simple and consistent interface optimized for common use cases. Provides clear and actionable information about user errors.

• Modular and configurable

Keras models are manufactured by connecting configurable building blocks to each other, with few restrictions.

• Easy to extend

Write custom building blocks to express new ideas for research. Create new layers, metrics, loss functions and develop state-of-the-art models


3. Process

• Read all images in numerical arrangement, at the same time you should normalized the data

• By using OneHotEncoder () transform the outputs and divide training and test data in 80% and 20% respectively.

• Create the different neural network configurations and train them

• With the use of matrix_confusion () determinate the precision of each neural network with respect to the test data 

• The report is made with the use of the ‘reportlab’ library. The ’matplotlib’ library was used to make the graphs

4. Analysis 

• Determine which neural network configuration was most effective and which transfer functions had the best results.

Since all settings use the same data for their training, they all have 1024 neurons in the input layer and 46 neurons in the output layer. In all the configurations, the activation function ‘relu’ was used, since in the development of the project it was determined that it was the one with the best results.

-The first neural network has an intermediate layer with 256 neurons

-The second neural network was configured with an intermediate layer with 2048 neurons

-The third neuron was configured with 3 intermediate layers with 256 neurons each one.

The second configuration despite having a single layer compared to the third configuration, has  a better result beacause it reach up  93.99% accuracy in comparation with the  92.93% and 92.43% accuracy of the first and third configuration respectively

• How the number of neurons affects the given problem? 

Between the configuration of network 1 and configuration 2, as they have the same number of layers, it can be seen that the more neurons we have, the better precision we obtain. However, it should be noted that for this configuration to have better precision, it is necessary that the number of epochs 'epochs' must be greater. While with fewer neurons, it returns good results with few epochs, as you increase its precision it does not improve.

• What data normalization strategy you used and what was the reason for selecting that strategy.

With the image represented as an array of numbers in a range between 0 and 1, the intermediate values represent the shadows or outlines of the symbol that it represents. We proceeded to round all the numbers in this matrix, in order to simulate that we have a fully black and white image and not a grayscale (shadow)

5. Results

6. Conclusion

• The best multilayer neural network structure with forward connection was determined to perform pattern classification tasks
• In addition, the information preprocessing stage (data cleaning, normalization, hot encoding, etc.) was applied correctly to obtain better results in the learning process.

7. Biography

Keras | TensorFlow Core. (s. f.). TensorFlow. Recuperado 14 de junio de 2020, de https://www.tensorflow.org/guide/keras?hl=es-419

Piloto-Rodríguez, R. (2017). Redes Neuronales Artificiales. Conceptos básicos y algunas aplicaciones en Energía. https://doi.org/10.13140/RG.2.2.23326.54083