Solution to tutorial 1
Problem description
Three problems that cannot be solved using traditional computing model but can be solved using neural computing include sensor processing, pattern recognition and speech and even text processing. It is because neural networks provide a different approach for data analysis and pattern recognition within the same data as compared to traditional computing approach. Text recognition, sensor processing, pattern recognition and speech recognition are all problems that cannot be well characterized. Therefore, traditional computing can only work well for problem that can be well described in terms of characteristics. These features include balancing checkbooks, keeping tabs of inventory, keeping ledgers among others (Derry & Lajoie, 2013).
Justification
In real sense, nearly all problems that can be well characterized do not require unique characteristics experienced in neural networks. Also, the processing style required for the problems of neural network mentioned above is parallel as opposed to sequential processing of well-characterized data by traditional computing. In addition, neural network will require right brained (gestalt) functions by means of images, pictures and controls to solve the problems mentioned above. On the other hand, traditional computing use left brained (logical) functions through rules, concepts and calculations. They are not application at all in the solution of the above problems(Derry & Lajoie, 2013).
Solution to tutorial 2
Description of Neural Network Paradigms
Learning paradigms such as supervised, unsupervised and hybrid sum-of-products are used to solve most problems of neural network. Supervised learning is an approach whereby both input as well as output of the neural network system is given. An artificial neural network is then employed to model the association that exists amid input and output. Unsupervised learning is where both the data and the cost function are given. The Artificial neural network is then trained in order to minimize the cost function by means of determining the appropriate relationship for both input and output. The hybrid sum-of product is an enhancement of both sums of product and back propagation paradigms. In this paradigm, the input layer is linked to the upper layer in a standard propagation approach. The next consecutive layers are then linked using the approach of sum of sum-of-products. Supervised learning paradigm can best be used to solve the problems mentioned in tutorial one above (Lamm, Decety & Singer, 2011).
Justification
Supervised learning is the best in this case given that it is suitable where the network is expected to reproduce characteristics of a given relationship. In our case, we are to look at the relationship that exists between inputs and output model of the neural network. For instance, a one to one association exits between the input text and output text elements which may lead to the reconstruction of the associated pattern of text from the ideal or the corrupted input. Either pattern or auto association category of learning can be employed for this solution. Supervised learning can also be applied for speech and gesture recognition (Lamm, Decety & Singer, 2011).
Solution to tutorial 3
Proprietary and nonproprietary Neural Network software models
Proprietary
Neuro model (NM) is a proprietary neural network software model. It is a window based artificial neural network (ANN) development package. This package does not require any scientific skills associated with artificial neural network. It performs better with real world datasets. The clustering algorithm for proprietary software offers dependable and reliable information for training when employed for pre-processing of all data sets. The internal features consist of different combination of mathematical techniques. The dynamic nonlinearity modeling along with implemented time characteristic allows the package to carry out online prediction of process situations in complex environments (Boehm, Abts & Chulani, 2000, Olsen & Sætre, 2007).
The cost function of this model is also minimized as it allows estimation of parameters accordingly. This cost function involves the overall accuracy criterion which is high in this model. It thus implies that the higher the degree of accuracy, the lower the cost functions. Some of the proprietary predictive software for data analysis includes RapidMiner, FICO predictive analytics, Salford analytics, and oracle data mining (ODM), MATLAB, Minitab and many others. The use of proprietary software saves almost 25 percent of the time and cost needed to complete the project. Proprietary software is therefore the best for option for various small and medium sized enterprises. The complete control of proprietary software makes it possible to maintain flexibility as well as being dynamic. Also, to match the requirements of the customer or client at any time (Boehm, Abts & Chulani, 2000, Olsen & Sætre, 2007). The Artificial neural network is then trained in order to minimize the cost function by means of determining the appropriate relationship for both input and output.
Non-proprietary
Unified modeling language (UML) is a non-proprietary model suitable for visualizing, constructing, documenting and specifying artifacts of software intensive system. It provides a standard approach for writing the blue print of the system. It can be used to model the objects and also as a specification language for software engineering. It can be used to generate a conceptual model of the system by means of standard graphical notation. The adaptive neuro fuzzy model for software development effort estimation has the capability of learning as well as interpretation. Each attribute in this model represents a factor that contributes to the effort of development as the application domain experience. It is the efficient method of cost effort estimation as compared to direct and non-algorithmic approaches (Boehm, Abts & Chulani, 2000, Preth, Shivakumar & Balaji, 2014).
Solution to tutorial 4
Explanation of pre and post processing techniques
Pre-processing techniques are those methods used for data transformation and input variable selection for the purpose of improving the model accuracy as well as to obtain the right scale for values to be used. Post processing techniques involves all methods used in the manipulation of the pre-processed data into information. In essence, output dataset is post-processed by returning it to its original scale and the forecast values are produced by the trained neural network. The capability of a pre-processing method to transform the pre-processed data into its original scale is what is called post-processing (Asadi et al., 2012; Chau, 2001).
Description of various pre and post processing techniques
Normalization process and data scaling are some of the examples of pre-processing techniques. Normalization is carried out for the main purpose of eliminating data redundancy from a list of data items and attributes available. Some of the normalization algorithms include Min-Max, Z-score and sigmoid normalization. Scaling on the other hand ensures that all values used are within the acceptable range before they are entered into the computer using the software in question (Asadi et al., 2012).
Justification
The pre-processing techniques such as normalization are important because it plays a big role in eliminating data redundancy from the available sets of raw data for easy handling. Normalization is also done in order to maintain data integrity. Scaling of values is also important as a pre-processing technique to ensure that all values are accurate. Also ensures that values are within the acceptable range before they are finally entered into the computer for final processing (Asadi et al., 2012).
Solution to tutorial 5
Data collection methods
In neural networks, the structure for the recognition of natural scene text is used. The structure does not need any human labeled data and more so carries our word recognition on the entire image. It makes the character based recognition system of the past obsolete. Synthetic text generation engine are used to generate data for training of the network model. This data generated is highly reasonable and adequate for the replacement of real data, resulting to the availability of infinite quantity of training data. Other words recognition models like dictionary encoding, character, sequence encoding and bag-of-N-grams encoding are also considered as ways of reading words in a different manner. In cases of language based with entirely unrestrained text recognition, the performance can be improved by means of standard datasets. It ensures zero costs for data acquisition by use of fast and simple machinery. A number of datasets began as data capture methods which intended to represent the visual world. They include Corel world, the Caltech-101 world and others (Jaderberg et al, 2014, Torralba & Efros, 2011). Synthetic text generation engine is most appropriate for the solution of the problems mentioned in tutorial 1 above.
Justification of method(s)
Synthetic text generation engine will be the most suitable for the solution of neural network problems since it is more accurate as compared to the normal statistical methods of data collection. As a result, the data generated by this method is extra reliable and sufficient for the replacement of real data. Speech recognition, pattern and text recognition are also complex neural network problems with no well-defined characteristics and hence they need more accurate and highly reliable methods of data collection for network training.
Solution to tutorial 6
Neural Network Project Management differs from traditional project
Justification
Neural network project based approach is essentially non-deterministic local search technique. It is different from the traditional project management which is very effective for global search. Neural network is also an intensification approach as opposed to traditional project management that is more of diversification approach. The next difference is that the neural network is a problem space based while traditional is the solution space based approach. It means that the problem parameters in neural network are repeatedly agitated to the next iteration as the similar heuristic is employed. For the case of solution based approach in traditional project, the solution is perturbed from one repetition to the other by employing the similar mechanism. Unlike traditional project management, neural network project weight vector which is adjusted for every repetitive execution (Agarwal, Colak & Erenguc, 2011).
The weighted problem parameters are also used in place of the initial parameters or constraints. Fresh solutions are normally produced for every repetitive execution in a neural network when the same heuristic are used. The search process is, therefore, guided by the weight adjustment guide approach. The selected priority rule in neural network is used several number of times and the best solution saves after a number of trials to be the concluding solution. As mentioned earlier, neural network technique is fundamentally a local search method given that the agitated vector generates an agitated activity list in the local neighborhood of the initial activity list. Another difference is that switching from traditional encoding to neural network is a challenge.. It is unlike the switch from neural network encoding which is straight forward as the very first procedure is just to order the activities by use of weighted parameters (Agarwal, Colak & Erenguc, 2011).
Last but not least, neural network based method employs LFT as the priority rule. In this case, the stopping condition is to stop if the solution is the same as the lower limit which is the critical path computed without any resource constraints. The stop condition also applies if the predetermined number of maximum schedules is reached. The learning rate is also set to a given value, say, 0.05 with the weights being initialized at 1 and adjusted after every repetition. For the case of traditional project, the two-point crossover and mutation probability is set at say, 0.5, and with a population size used (Agarwal, Colak & Erenguc, 2011).
Solution to lab 1 scenario
Appropriate Neural Network Model
The appropriate modeling to use is artificial neural network (ANN) modeling. Artificial neural network offers flexible computing structure for modeling a wide range of problems associated with nonlinear. The problem to be solved in this case is predicting a decrease or increase in the market value of one bedroom flats. The problem itself is characterized with features of nonlinear variables identified by location, heating system garage and years of release. As a result ANN becomes the appropriate model to apply in this case. It will be more advantageous to use ANN instead of other classes of the nonlinear model since it is a universal approximator. It is therefore capable of approximating large category of functions with a high level of precision due to its inherent power of parallel processing. In addition, the network model is largely determined by attributes of data and hence no prior model assumption is needed in the process of building the artificial neural network model (Asadi, 2012).
Pre and post processing techniques
The pre-processing techniques data transformation approaches such as scaling. It will aid in the conversion of any larger values within the data into an acceptable range for coding using the analysis software. Data classification is also part of pre-processing activity that is to be carried out in this case. The classification of data attributes is under four categories namely location, heating system, garage and years of release (Pradhan & Lee, 2010). In addition, the network model is largely determined by attributes of data and hence no prior model assumption is needed in the process of building the artificial neural network model
In the post processing involves further processing of the datasets with the aim of regaining the initial scale. The techniques used here include computation analysis such as prediction of either increase or decrease in market value of bedroom flats. The generation techniques are also used in order to generate the predicted values (Pradhan & Lee, 2010).
Data collection
The following shows how row data can be generated according to the features identified as house location, heating system, garage, and years of the lease. It is done by the use of MakeData method. We can let house location to be location, heating system to be heating, garage to remain as garage and years of the lease to be years. Then we can proceed as follows (Pradhan & Lee, 2010).
Create 22 arbitrary weights between -2.0 and +2.0
create a 4-5-3 neural network
load weights into the neural network
open a result file for writing
loop 50 times
generate four random inputs x0, x1, x2, x3 between 1.0 and 9.0
compute the y0, y1, y2 neural outputs for the input values
determine largest of y0, y1, y2,
if y0 is largest write x0, x1, x2, x3, location
else if y1 is largest write x0, x1, x2, x3, heating
else if y2 is largest write x0, x1, x2, x3, garage
else if y3 is largest write x0, x1, x2, x3, years
end loop
close result file
Categorizing data for training and testing
First, you determine the number of rows in which the data can be split into and then create a matrix to hold all the data. You then read a line of data repeatedly as you assign them to fields for every numeric input. Perform repeated normalization for input data so that they range between -1.0 and +1.0 for every row in the matrix. Then encode categorical output repeatedly using 1-of-N as well as placing normalized and encoded data in a matrix again repeatedly until you are through with all rows in the matrix. Shuffle the matrix and then create train and test matrices. In our case, we can first transfer 16 rows (almost 80% of total rows) of the matrix to training and the remaining rows to testing. The pseudo code for this training and test data categorization is as follows (Pradhan & Lee, 2010).
decide how many rows to split data into
generate a matrix to hold all data
loop
read a line of data
parse each field
foreach numeric input
normalize input to between -1.0 and +1.0
endforeach
encode categorical output with 1-of-N
put normalized and encoded data in the matrix
end loop
shuffle matrix
create train and test matrices
move first 80% rows of the matrix to train, outstanding rows to test
Training the network
The first thing to do is to create an empty neural network. Then carry out the following steps in a looping sequence. Start by generating the candidate set of weights followed by loading weights into the neural network. Then compute the neural output, cross entropy error and total error in that order for every training vector. Then save the current weights if they are best found. All these procedures are repeated until some stopping condition is achieved. You finally return the best weight found (Pradhan & Lee, 2010).
References
Agarwal, A., Colak, S., & Erenguc, S. (2011).A neurogenetic approach for the resource-constrained project scheduling problem.Computers & Operations Research, 38(1), 44-50.
Asadi, S., Hadavandi, E., Mehmanpazir, F., & Nakhostin, M. M. (2012).Hybridization of evolutionary Levenberg–Marquardt neural networks and data pre-processing for stock market prediction.Knowledge-Based Systems, 35, 245-258.
Boehm, B., Abts, C., & Chulani, S. (2000). Software development cost estimation approaches—A survey. Annals of Software Engineering, 10(1-4), 177-205
Chau, T. (2001). A review of analytical techniques for gait data. Part 2: neural network and wavelet methods. Gait & Posture, 13(2), 102-120.
Derry, S. J., & Lajoie, S. P. (2013). A Middle Camp for (Un) lnte|| igent Instructional Computing: An Introduction. Computers as cognitive tools, 1.
Jaderberg, M., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014).Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition.arXiv preprint arXiv:1406.2227.
Lamm, C., Decety, J., & Singer, T. (2011). Meta-analytic evidence for common and distinct neural networks associated with directly experienced pain and empathy for pain. Neuroimage, 54(3), 2492-2502.
Olsen, K. A., & Sætre, P. (2007). ERP for SMEs–is proprietary software an alternative?. Business Process Management Journal, 13(3), 379-38
Pradhan, B., & Lee, S. (2010). Landslide susceptibility assessment and factor effect analysis: backpropagation artificial neural networks and their comparison with frequency ratio and bivariate logistic regression modelling. Environmental Modelling & Software, 25(6), 747-759.
Preth. R, Shivakumar.N & Balaji, N. (2014). Software Effort Estimation Using Attribute Refinement based Adaptive Neuro Fuzzy Model. Thiagarajar College of Engineering, Madurai, India, KLN College of Engineering, Madurai, India, Volume 3, Special Issue.
Torralba, A., & Efros, A. A. (2011, June) .Unbiased look at dataset bias. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on (pp. 1521-1528). IEEE.
