Jekyll2024-01-23T12:13:08+00:00https://faizan-e-mustafa.github.io/feed.xmlFaizan E MustafaPortfolioFaizan E Mustafafaizanmustafa750@gmail.comEmotion Analysis2021-04-08T00:00:00+00:002021-04-08T00:00:00+00:00https://faizan-e-mustafa.github.io/EmotionAnalysis<p>This course Emotion Analysis is taught by <a href="https://www.romanklinger.de/">Dr. Roman Klinger</a> at University of Stuttgart. More information can be found on the <a href="https://www.emotionanalysis.de/">course website</a>.</p>
<p>This post contains the presentation slides for the assignments done during the course. Assignments can be found <a href="https://www.emotionanalysis.de/assignments/">here</a>.</p>
<table>
<thead>
<tr>
<th>Assignment No.</th>
<th style="text-align: left">Assignment</th>
<th style="text-align: center">Slides</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td style="text-align: left">Corpus Creation</td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/EmotionAnalysis/Assignment_1.pdf">link</a></td>
</tr>
<tr>
<td>2</td>
<td style="text-align: left">ML-based Emotion Classification</td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/EmotionAnalysis/Assignment_2.pdf">link</a></td>
</tr>
<tr>
<td>3</td>
<td style="text-align: left">Literature Review</td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/EmotionAnalysis/Assignment_3_paper.pdf">summary</a> / <a href="https://Faizan-E-Mustafa.github.io/pdfs/EmotionAnalysis/Assignment_3_slide.pdf">slide</a></td>
</tr>
<tr>
<td>4</td>
<td style="text-align: left">Role Labeling</td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/EmotionAnalysis/Assignment_4.pdf">link</a></td>
</tr>
</tbody>
</table>Faizan E Mustafafaizanmustafa750@gmail.comEmotion Analysis from text course taught at University of Stuttgart.Research Seminar2021-04-08T00:00:00+00:002021-04-08T00:00:00+00:00https://faizan-e-mustafa.github.io/ResearchSeminar<p>Research seminar is taught by <a href="http://www.schulteimwalde.de/">Dr. Sabine Schulte im Walde</a> at University of Stuttgart. Follwing are the main objetives of this seminar.</p>
<ul>
<li>Introduce students to different reserach groups at <a href="https://www.ims.uni-stuttgart.de/">IMS</a>.</li>
<li>Guidance regarding thesis.</li>
<li>Improve (poster) presentation skills.</li>
</ul>
<p>Following are the materials created during this seminar.</p>
<h2 id="poster-presentation">Poster Presentation:</h2>
<p>The poster based on the research paper named <a href="https://ieeexplore.ieee.org/document/6949798">A study on emotion recognition from body gestures using Kinect sensor</a> can be downloaded from <a href="https://Faizan-E-Mustafa.github.io/pdfs/ResearchSeminar/poster.pdf">here.</a></p>
<h2 id="talk">Talk:</h2>
<p>The presentation slides are completely based on the <a href="https://huyenchip.com/2020/01/18/tech-workers-19k-compensation-details.html">this</a> blog post. The slides are available <a href="https://Faizan-E-Mustafa.github.io/pdfs/ResearchSeminar/slides.pdf">here</a> and presentation video can be found <a href="https://www.youtube.com/watch?v=_uqSZj0N8sA">here</a>.</p>Faizan E Mustafafaizanmustafa750@gmail.comResearch Seminar taught at University of Stuttgart.Statistical Dependency Parsing2021-04-07T00:00:00+00:002021-04-07T00:00:00+00:00https://faizan-e-mustafa.github.io/SDP<p>This seminar <code class="language-plaintext highlighter-rouge">Statistical Dependency Parsing(Winter 2020)</code> was taught at University of Stuttgart by <a href="https://www.ims.uni-stuttgart.de/en/institute/team/Falenska/">Dr. Agnieszka Faleńska</a> and <a href="https://www.ims.uni-stuttgart.de/en/institute/team/Yu-00010/">Mr. Xiang Yu</a>. This post contains material produced during this seminar.</p>
<h2 id="dependency-parser-project">Dependency Parser Project:</h2>
<p>Following parsers were implemented from scratch .</p>
<ul>
<li>Graph based: Eisner</li>
<li>Transition based: Arc-standard (Not Complete yet)</li>
</ul>
<p>if you want to know more about the project you can read more about it <a href="https://Faizan-E-Mustafa.github.io/pdfs/SDP/SDP_Report.pdf">here</a>. The repository containing the code can be accessed using <a href="https://github.com/Faizan-E-Mustafa/Dependency-Parsing">this link</a>.</p>
<h2 id="paper-summaries">Paper Summaries:</h2>
<p>We had to summaries atleast two papers.</p>
<table>
<thead>
<tr>
<th>Paper</th>
<th style="text-align: center">Link</th>
<th style="text-align: center">Summary</th>
</tr>
</thead>
<tbody>
<tr>
<td>Frustratingly Easy Cross-Lingual Transfer for Transition-Based Dependency Parsing</td>
<td style="text-align: center"><a href="https://www.aclweb.org/anthology/N16-1121.pdf">link</a></td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/SDP/SDP_Summary2.pdf">link</a></td>
</tr>
<tr>
<td>Generalized Transition-based Dependency Parsing via Control Parameters</td>
<td style="text-align: center"><a href="https://www.aclweb.org/anthology/P16-1015/">link</a></td>
<td style="text-align: center"><a href="https://Faizan-E-Mustafa.github.io/pdfs/SDP/SDP_Summary1.pdf">link</a></td>
</tr>
</tbody>
</table>
<h2 id="presentation">Presentation:</h2>
<p>I presented a paper named <a href="https://www.aclweb.org/anthology/P19-1237/">Graph-based Dependency Parsing with Graph Neural Networks</a>. The presentation can be downloaded from <a href="https://Faizan-E-Mustafa.github.io/pdfs/SDP/SDP_Presentation.pptx">here.</a></p>Faizan E Mustafafaizanmustafa750@gmail.comStatistical Dependency Parsing seminar taught at University of Stuttgart.Cross-lingual gender prediction with multi-lingual embeddings and linguistic features2021-01-01T00:00:00+00:002021-01-01T00:00:00+00:00https://faizan-e-mustafa.github.io/Team-Lab<h3 id="abstract">Abstract:</h3>
<p><em>Most systems for gender profiling have focused on mono-lingual use-cases. In this paper we implement two systems for cross-lingual gender prediction and compare them: One system is based on linguistic features while the other leverages a multi-lingual embedding approach (XLMRoBERTa). Moreover, we analyse which linguistic properties are most predictive for gender profiling across languages. We find that XLM-RoBERTa performs best with accuracy scores of up to 0.87. Classification on top of linguistic features did not consistently generalize cross-lingually. However, linguistic feature analysis supported previously observed divergences of male and female language use across languages.</em></p>
<p>The research paper is avaliable <a href="https://Faizan-E-Mustafa.github.io/pdfs/TeamsLab/xlingual_gender_prediction.pdf">here</a>.</p>Faizan E Mustafafaizanmustafa750@gmail.comTeams Lab Project Report: Cross-lingual gender prediction with multi-lingual embeddings and linguistic featuresKeras Implementation of Image Captioning Model2019-09-29T00:00:00+00:002019-09-29T00:00:00+00:00https://faizan-e-mustafa.github.io/ImageCaptioning<p>Image captioning is a task that involves computer vision and natural language processing. It takes an image and can describe what’s going on in the image in plain English. A CNN architect is used to extract features from the images. Then the encoded image is passed through a decoder. Since RNN is very good with sequential data and we need to describe an image in a sentence so we could use an RNN or its variant as a decoder.</p>
<h2 id="data-set">Data set:</h2>
<p>I used flicker8k data set. It contains 8,000 images. Each image has five captions. The model was trained on 6,000 images, and 1,000 images were spared for Dev and Test set each.</p>
<h2 id="encoder">Encoder:</h2>
<p>I used InceptionV3 as an encoder. Taking advantage of the transfer learning pre-trained weights of inception V3 was used. The second last layer of the inception model was the new output of the model, as we want to encode an image into a vector.</p>
<h2 id="decoder">Decoder:</h2>
<p>I used LSTM as a decoder. The model summary is as below.</p>
<ul>
<li>This is the image model as it takes the encoded image from the inception model and uses a fully connected layer to scale it down from 2048 dimensions to 300 dimensions.</li>
</ul>
<figure class="">
<img src="/images/posts/ImageCaptioning/ImageModel.png" alt="this is a placeholder image" />
<figcaption>
Image Model
</figcaption></figure>
<ul>
<li>This is the language model; It receives one hot vector that is first converted into the embedding vector and then passed through bidirectional LSTM.</li>
</ul>
<figure class="">
<img src="/images/posts/ImageCaptioning/LangModel.png" alt="this is a placeholder image" />
<figcaption>
Language Model
</figcaption></figure>
<ul>
<li>The image model and language model are then merged and passed through another bidirectional LSTM to output a vector of vocabulary size. The highest probability value of the vocabulary vector is the next predicted word.</li>
</ul>
<figure class="">
<img src="/images/posts/ImageCaptioning/Final.png" alt="this is a placeholder image" />
<figcaption>
Final Model
</figcaption></figure>
<p>These were the values of different variables.</p>
<p>Embedding size = 300</p>
<p>Vocabulary size= 8256</p>
<p>Dropout = 0.5</p>
<p>Optimizer = Adam</p>
<p>Batch Size = 128</p>
<h2 id="training">Training:</h2>
<p>I trained the model for 13 epochs. The following are loss values for some epochs.</p>
<p>11 epochs : 2108s 703ms/step - loss: 1.3744 - acc: 0.6058</p>
<p>12 epochs : 2106s 703ms/step - loss: 1.2529 - acc: 0.6332</p>
<p>13 epochs : 2097s 700ms/step - loss: 1.1560 - acc: 0.6571</p>
<p>It gave decent results at the 12th epoch. But the results deteriorated a bit at the 13th epoch, so I did not train it further.</p>
<p><strong>Note: I trained my model for 10 epochs and the model was working fine. But the next day model was giving results of an untrained model. Turned out I made a new vocabulary and overwrote the previous one. Every time you tokenize a new vocabulary, it assigns a new index to the same word. So I had to train my model all over again. It will make more sense when you go through the code.</strong></p>
<h2 id="testing">Testing:</h2>
<p>I used a greedy search and beam search to predict the results. Unlike greedy search where we pick only the highest probability to get the next word prediction, In beam search, we pick as many words as we like each time it makes a prediction. Suppose we picked three words, then we will take into account the probability of a whole caption by using the first word, second word, and the third word separately. Whichever is giving us the maximum probability for the caption, we pick that one word.</p>
<p>If you want to know more about the beam search, you can find Andrew Ng’s video <a href="https://www.coursera.org/lecture/nlp-sequence-models/beam-search-4EtHZ"><strong><em>Here.</em></strong></a></p>
<h2 id="results">Results:</h2>
<figure class="">
<img src="/images/posts/ImageCaptioning/result.png" alt="this is a placeholder image" />
</figure>
<p>Sometimes Beam Search does its trick.</p>
<figure class="">
<img src="/images/posts/ImageCaptioning/Beam.png" alt="this is a placeholder image" />
</figure>
<p>Kindly share your reviews about the project. Thanks</p>Faizan E Mustafafaizanmustafa750@gmail.comImage captioning is a task that involves computer vision as well as Natural language processing. It takes an image and is able to describe whats going on in the image in English. It uses InceptionV3 to extract features from images and LSTM to generate captions for images.This implementation uses Keras with Tensorflow back end.Study Data Science in Germany - A Guide for Masters Aspirants2019-09-27T00:00:00+00:002019-09-27T00:00:00+00:00https://faizan-e-mustafa.github.io/ProjectGermany<p>Germany is one of the most coveted place for students who want to study abroad . High quality education in a very low financial cost is one the major reasons that every year a huge amount of undergraduate and graduate students apply for their desired programs . Germany offers a lot of opportunities for students who want to pursue their studies in every possible field. Like many other fields , Germany also offers best quality education for students who want to study Data Science , Artificial Intelligence , Machine Learning and related subjects .</p>
<p>After completing my under-graduation , I was willing to do Masters from abroad in a field related to Data Science and Machine Learning . After doing a lot of research, Germany came out to be the obvious choice for me. I applied and got admission in University of Stuttgart for MSc Computational Linguistics Course . The application process was not easy and i also faced some difficulties due to lack of information related to my subjects. So, i wanted to help people who want to study in Germany and could not find relevant information .</p>
<h3 id="msc-data-science-machine-learning-germany-group">MSc Data Science/ Machine Learning (Germany) Group:</h3>
<p>When I was searching for universities that offered programs related to my desired subjects , I was not able to find information in one place . I believe that solution to problems could be found through team work . So I created a group on a social website where anyone can share and get information about the masters programs . This synergy will help everyone that is why motto of group is “Help yourself by helping others”.</p>
<p><a href="https://www.facebook.com/groups/364661040940901/"><strong><em>Here is link</em></strong></a> to the MSc Data Science/ Machine Learning (Germany) Group.</p>
<figure class="">
<img src="/images/posts/ProjectGermany/a.jpg" alt="this is a placeholder image" />
<figcaption>
Moto of Group
</figcaption></figure>
<h3 id="whats-in-that-group-">Whats in that Group ?</h3>
<ul>
<li>
<p>I have made a list of courses that are offered in different Universities in Germany . It is composed of Masters programs in the fields of Data Science , Machine Learning and other related fields.</p>
<p><a href="https://docs.google.com/spreadsheets/d/1AfoKjQJnyRQgC1SXQ3mWDkiOlsiw1hShDlBr7v-YonY/edit?fbclid=IwAR2CFQwxvO8bR7R1ERYxFGBmdKO-VKAVbn2-W4pPpRtDOn1vKAS_9mc0LaY#gid=0"><strong><em>Here is Link</em></strong></a> to the List.</p>
</li>
<li>
<p>I have recently complied a list of most useful links from different resources which will be helpful for people at different stages of their application phase . As i am familiar with the application procedure from Pakistan , i do not know if the procedure is same for reader’s country or not .</p>
<p><a href="https://www.notion.so/Project-Germany-9f07122d9f5547e59d5edb85aa0c5c9b"><strong><em>Here is Link</em></strong></a> to the Website:</p>
</li>
</ul>
<figure class="">
<img src="/images/posts/ProjectGermany/b.png" alt="this is a placeholder image" />
<figcaption>
Screen Shot of Website
</figcaption></figure>
<p>If you need any help , feel free to contact me.</p>Faizan E Mustafafaizanmustafa750@gmail.comA guide for people who aspire to do Masters in Data Science / Machine Learning or related field from Germany.Batch Vs Mini Batch Gradient Descend2019-09-12T00:00:00+00:002019-09-12T00:00:00+00:00https://faizan-e-mustafa.github.io/GradientDescend<p>Deep leaning works best in the regime of Big data . Algorithms tends to work slow on large data sets. that’s why we need fast optimization algorithms to cope well with the large data sets. One little example of optimization is when we use vectorization instead of loops . So whenever we find a loop that can be implemented using vectorization we do it without giving it a second thought.</p>
<h3 id="batch-vs-mini-batch-gradient-descend-">Batch vs Mini batch gradient descend :</h3>
<p>Mini batch gradient descend is also used to get an optimized algorithm.</p>
<p><strong>We want our gradient descend(backward propagation ) to not sit idle until we have gone through whole training data (forward propagation).</strong></p>
<p>Unlike batch descend , mini batch descend use mini batches of training data . when we use whole training data to get the output from forward propagation .Gradient descend step will be at rest during this whole time . If we could somehow manage to get this gradient descend working before going through whole training data we could get a better optimized algorithm. So we divide training data into mini batches and we do our forward and back propagation for each mini batch .</p>
<p><strong>Batch gradient descend :</strong> If batch size is equal to # of training examples(m)</p>
<p><strong>Disadvantage:</strong> It takes very long time per iteration if training set is very large. Gradient descend sits idle.</p>
<p><strong>Stochastic gradient descend:</strong> If batch size is 1 then we will do forward and backward propagation for 1 training example at a time.</p>
<p><strong>Disadvantage :</strong> We lose speed up from the vectorization because we are using only a single training example at a time.</p>
<p><strong>What should be the batch size ?</strong>
We want to keep vectorization and at the same time we want to make progress without going through entire training set. so we have to use batch size between 1 and m . typical mini batch size are 64,128,256 . These sizes are usually written as power of 2.</p>
<p><strong>Note:</strong> If training set is small ( m < 2000) then we can use batch gradient descend because in case of small training set gradient decent will not have to wait long enough.</p>Faizan E Mustafafaizanmustafa750@gmail.comDifference between Batch and mini-batch gradient descend and their advantages and disadvantages.