Problem 2: Recurrent Neural Networks
Recurrent neural networks (RNNs) are deep-learning architectures which are particularly
powerful in dealing with time-series (e.g., financial) and natural language data. In this problem,
you are going to write a program to simulate the basic forward propagation mechanism in an
RNN using simple 1D arrays. Note that you do not need to understand what an RNN is to
solve this problem; you will find all the necessary information that you need to solve this
problem in the following description.
An RNN cell at time t takes an input x t and a hidden state h t from the previous cell, and
computes a next state h t +1 and an output o t .
Specifically, each RNN cell computes o t and h t +1 using the following two functions: o t = sigmoid(0.1 x t + 1.5 h t )
h t +1 = tanh(0.5 x t − 2 h t )
where
sigmoid( x ) = 1/(1 + e − x )
tanh( x ) = 2 × sigmoid(2 x ) − 1
and e = 2.72 .
Multiple RNN cells can then be connected to form a network. The following figure depicts an
RNN of T cells that accepts an input sequence x 0 , x 1 ,..., x T −1 and an initial hidden state h_0
as inputs, and outputs a sequence o 0 , o 1 , ..., o T −1 .
Write a C++ program that implement the above simple RNN.
Input:
• the first line contains two numbers: an integer T denoting the recurrent times (
1 ≤ T ≤ 100 ) and a floating-point number h 0 denoting the initial hidden state;
and
• the second line contains the input sequences which are T floating-point numbers
x 0 , x 1 ,..., x T −1 .
Output:
• Your program should display the hidden state sequences h 1 , h 2 , ..., h T in the
first line and the output sequences o 0 , o 1 , ..., o T −1 in the second line. • Floating-point numbers are displayed with 10 decimal places. Use
setprecision(n) defined in the header <iomanip> to set the precision
parameter of the output stream.
Requirement:
• You will need to complete the following functions in the provided template
2.cpp . Read the code in the template carefully to see what have been provided
for you. You will find details of the function prototypes in the in-code comments
too.
◦ sigmoid() for sigmoid activation function
◦ tanh() for tanh activation function
◦ ComputeH() for computing the next hidden value in RNN cell
◦ ComputeO() for computing the output value at current time step
◦ PrintSeqs() for printing the values in a 1D array to screen
◦ main() for main function
• Use 1D arrays to store the sequences for x i , h i and o i .
• You can ONLY use the simple data types char , bool , int , double and
arrays. In other words, you are not allowed to use other data types or data
structures such as strings or STL containers (e.g., vectors), etc.
Sample Test Cases
User inputs are shown in blue .
2_1:
1 1.0
1.0
-0.9053193842
0.8321596401
2_2: 5 0.0
1.0 2.0 3.0 4.0 5.0
0.4623655914 0.0751742883 0.8741722520 0.2466235712 0.9645899021
0.5249949450 0.7097382221 0.6018123354 0.8471395064 0.7048466172
2_3:
3 1.0
0.0 0.0 0.0
-0.9641167571 0.9586890814 -0.9578009493
0.8177157977 0.1904499845 0.8082908051