WebHome < Estrinfo

---+!! Web and Social Information Extraction - A.Y. 2019/2020

The course presents algorithms and architectures to retrieve information from the web and to analyze social networks. Topics are information retrieval, web search engines, web mining, social network analytics.
| *Instructors*  |  *Telephone*  | *Office hours* | *Studio* |
| <p> [[http://www.dsi.uniroma1.it/~velardi/index.html][Paola Velardi]]</p> <p> </p> <p> [[mailto:prenkaj@di.uniroma1.it][Bardh Prenkaj]]</p> <p>(teaching assistant)</p>  | 06-49918356 | send e-mail | Via Salaria 113 - 3° floor n. 3412 |
---++ *IMPORTANT INFORMATION: From Monday, March 9th until the end of COVID-19 emergency, lessons will be REGULARLY held in streaming following the same schedule of in-class lessons. Students will receive the link to the videoconference few minutes before the beginning of the lesson. Information will be distributed through the course Google group and you will be able to participate in the videoconference ONLY using your institutional account. Videorecorded lessons and labs are available on a drive folder ONLY to students of the Google group.*
---++ Course schedule

II semester:

| *When* | |  *Where*  |

| Monday  | 14.00-16.00  | <p>aula alfa via Salaria 113</p> <p>(Lab will be held in Colossus</p> <p>labs- via salaria 113 - Since 2nd week of March)</p> |
| Wednesday  | 10.00-13.00  | aula alfa via Salaria 113 |

---++ Course Organization

The course presents architectures and algorithms related with the extraction of information from the Web, analyzing both Web Search engines and on-line Social Networks. A number of lessons are held in the Colossus lab.

During the lab, students will learn:
   * Lucene, an open-source text-search library. With Lucene, you will learn to index and search a document archive
   * How to useTwitter API, to track and index Twitter messages, to create and analyze word time series, and more
   * To build a web scraper and trace the content of a forum

---++ Self Assessment and Final Project

Self assessments and final project are sent to all students by email (google group).
---++ Summary of Course Topics

*Architecture of Information Retrieval systems*. Tokenisation, stop-word removal and stemming; morphology; selection of index terms, use of thesauri. Inverted indices. Boolean and vector-space retrieval models, ranked retrieval and text-similarity metrics. Performance metrics: recall, precision, and F-measure. Evaluations on benchmark text collections. Latent Semantic Indexing. Relevance feedback. Query expansion.

*Web Information Retrieval*. Browsing and Scraping. Link analysis: Page Rank and HITS.

*Social Network analysis*: Opinion Mining, Social Network analysis, Community Detection, Social Media Analytics, Recommender systems

---++ Textbooks

   * Ricardo Baeza-Yates, Berthier Ribeiro-Neto [[http://people.ischool.berkeley.edu/~hearst/irbook/index.html][Modern Information Retrieval]], !Addison Wesley Longman Publishing Co. Inc.
   * Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, [[http://nlp.stanford.edu/IR-book/][Introduction to Information Retrieval]], University Press. 2008.
   * David Easley and Jon Kleinberg: Networks, Crowds, and Markets: Reasoning About a Highly Connected World http://www.cs.cornell.edu/home/kleinber/networks-book/
   * Chen, Lakshmanan and castillo: Information and Influence Propagation in Social networks https://wiki.eecs.yorku.ca/course_archive/2014-15/F/4412/_media/social_networks.pdf

---++ Exam

Standard grading policy:
   * a) Written or oral exam on course material (50%). For those who attend classes, written test is split into a mid-term and a final test (usually, April and June).
   * b) Project (teams of 2) (50%). The quality of developed software *is matter of evaluation*.
%RED% *IMPORTANT: Due to COVID-19 emergency, the mid-term test will be replaced by a new grading policy, which includes an "interactivity grade" to prize active students (those who submit self assessments and lab exercises, and present their work during classes in streming).* %ENDCOLOR%

%RED% *The interactivity grade will be 1/3 of your final grade, and incorporates also interactivity and contributions during Labs.* %ENDCOLOR%

%RED% *Final grade will be computed as follows:* %ENDCOLOR%
   * %RED% *1/3 interactivity* %ENDCOLOR%
   * %RED% *1/3 project* %ENDCOLOR%
   * %RED% *1/3 final written test (hopefully in presence..)* %ENDCOLOR%
%RED% *Those with zero or low interactivity will be given the opportunity to recover the missing 1/3 (with a more complex written test ).* %ENDCOLOR%
---++ Project

The project is presented after the first 4-5 weeks of the course.

Find here, as an example, the best project in 2018: [[%ATTACHURL%/wsie_report_moschella_spini.zip][wsie_report_moschella_spini.zip]]:
---++ Project 2019

Find here the dataset for the project: [[%ATTACHURL%/datasetwebsocial2019.zip][datasetwebsocial2019.zip]]

The description of the project can be downloaded from this link: &lt;a data-saferedirecturl="https://www.google.com/url?q=https://docs.google.com/document/d/1NYVXM_X7qEX95RdFHnw0M3H2b-2BU_72IwmMxWVNca8/edit?usp%3Dsharing&source=gmail&ust=1559035338277000&usg=AFQjCNF9cfAdDVVAucDhoRLQa0NDtctAXw" href="https://docs.google.com/document/d/1NYVXM_X7qEX95RdFHnw0M3H2b-2BU_72IwmMxWVNca8/edit?usp=sharing" target="_blank"&gt;https://docs.google.com/document/d/1NYVXM_X7qEX95RdFHnw0M3H2b-2BU_72IwmMxWVNca8/edit?usp=sharing&lt;/a&gt;.

%RED% *best project in 2019* %ENDCOLOR%: [[%ATTACHURL%/MiningEvolvingTopics_best_2019.pdf][MiningEvolvingTopics_best_2019.pdf]]  (code can be requested to Bardh Prenkaj)
---++ Project 2018

The project is on Recommender Systems.

The description is provided in [[%ATTACHURL%/Project_Description_2018.pdf][Project_Description_2018.pdf]]

The description includes the link from where you can download the main dataset for the project, the Wiki_MID dataset.

The three additional datasets are: [[%ATTACHURL%/S21.tsv][S21.tsv]] [[%ATTACHURL%/S22_preferences.tsv][S22_preferences.tsv]] and [[%ATTACHURL%/S23.tsv][S23.tsv]]

---++ Project 2020

assigned (recommender Systems in Tourism)
---++ Google Group

<br />%TABLE{sort="off"}%
|  *Please Subscribe to Web and Social 2020 Group* |
| [[https://groups.google.com/a/di.uniroma1.it/forum/#!forum/webandsocial2020][Web and Social 2020 on Google Groups]] |

---++ Slides and course materials (note: *UPDATED = 2020*)

<table border="1" cellpadding="0" cellspacing="1"> <tbody> <tr><th style="text-align: left;">Last Update</th><th style="text-align: center;">Topic</th><th style="text-align: right;">PPT</th><th style="text-align: right;">ZIP/PDF</th><th>Details</th><th>Suggested readings</th></tr> <tr><th>2020</th> <td>Introduction</td> <td>

 [[%ATTACHURL%/1.WS1.introAB.pptx][1.WS1.introAB.pptx]] (part 2) [[%ATTACHURL%/1.WS1.introAB.pptx][<br />]]

 [[%ATTACHURL%/1.WS1.introB.pptx][1.WS1.introB.pptx]] (part 3)</td> <td>

 [[%ATTACHURL%/1.WS1.introA.pptx.zip][1.WS1.introA.pptx.zip]] (part 1) [[%ATTACHURL%/1.WS1.introcompressed.pdf][<br />]]

 </td> <td>Introduction, architecture of IR systems, text processing, indexing</td> <td>

 A survey on Indexing Techniques for Big Data: Taxonomy and Performance Evaluation (2015) https://www.researchgate.net/publication/273082158_A_survey_on_Indexing_Techniques_for_Big_Data_Taxonomy_and_Performance_Evaluation

 The case for learned index structures (April 30th, 2018) https://arxiv.org/pdf/1712.01208.pdf</td> </tr> <tr><th>2020</th> <td>Basic Ranking Models</td> <td> [[%ATTACHURL%/2.VectorSpaceModel.pptx][pptx]]</td> <td> [[%ATTACHURL%/2.VectorSpaceModel.pdf][pdf]]</td> <td>Basic ranking models: Boolean, Vector Space</td> <td>https://nlp.stanford.edu/IR-book/pdf/01bool.pdf</td> </tr> <tr><th style="text-align: left;">2020</th> <td>Query Expansion</td> <td style="text-align: left;"> [[%ATTACHURL%/3.QueryExpansion.pptx][ppt]]</td> <td> </td> <td>Improving basic IR models</td> <td>

 [[%ATTACHURL%/Query_Expansion_Techniques_for_Information_Retriev_1.pdf][Query_Expansion_Techniques_for_Information_Retriev_1.pdf]] [[%ATTACHURL%/QueryExpansionTechniquesSurvey.pdf][<br />]]

 [[%ATTACHURL%/QueryExpansionTechniquesSurvey.pdf][QueryExpansionTechniquesSurvey.pdf]]</td> </tr> <tr><th>2020</th> <td>

 Eigenvectors, eigenvalues and SVD decomposition

 Retrieval with Latent Semantic Indexing & Word Embeddings</td> <td> [[%ATTACHURL%/5b.Embeddings.pptx][5b.Embeddings.pptx]] [[%ATTACHURL%/5.LSI.pptx][<br />]]</td> <td style="text-align: left;">

 [[%ATTACHURL%/5.LSI.pptx.zip][5.LSI.pptx.zip]] [[%ATTACHURL%/5.LSIEmbeddings.pdf][<br />]]

 [[%ATTACHURL%/5.LSI.pdf.zip][5.LSI.pdf.zip]]

 [[%ATTACHURL%/5b.Embeddings.pdf][5b.Embeddings.pdf]]

 </td> <td>Alternative Ranking Models based on word similarities</td> <td>

 https://cs224d.stanford.edu/lecture_notes/notes1.pdf

 BERT: https://arxiv.org/pdf/1810.04805.pdf

 (video lessons available on drive for Google Group)</td> </tr> <tr><th>2020</th> <td>Evaluation</td> <td style="text-align: left;"> [[%ATTACHURL%/3.EvaluationIR.pptx][3.EvaluationIR.pptx]] [[%ATTACHURL%/4.EvaluationIR.pptx][<br />]]</td> <td> </td> <td>

 Performance measures and benchmarking of IR systems <br />

 </td> <td>

 https://papers.nips.cc/paper/5867-precision-recall-gain-curves-pr-analysis-done-right.pdf

 see also: https://trec.nist.gov/ for TREC evaluation challenges

 (video lessons available on drive for Google Group)</td> </tr> <tr><th>2020</th> <td>Web Search</td> <td style="text-align: left;"> </td> <td> [[%ATTACHURL%/6.WebSearch-compressed.pdf][6.WebSearch-compressed.pdf]]</td> <td>Web IR: crawling, scraping, searching on the Web; Categorization of web pages</td> <td>

 (video lessons available on drive for Google Group)

 [[%ATTACHURL%/WEB_SITE_CLASSIFICATION_FEATURES_AND_ALG.pdf][WEB_SITE_CLASSIFICATION_FEATURES_AND_ALG.pdf]] (survey on feature selection for web page classification)

 [[%ATTACHURL%/information-10-00150.pdf][information-10-00150.pdf]] (survey on machine learning for document classification )</td> </tr> <tr><th>2020</th> <td>Link Analysis</td> <td style="text-align: left;"> [[%ATTACHURL%/7.link_analysis.pptx][7.link_analysis.pptx]] [[%ATTACHURL%/7.link_analysis.pptx][<br />]]</td> <td> [[%ATTACHURL%/7.link_analysis-compressed.pdf][7.link_analysis-compressed.pdf]]</td> <td>Hyperlink-based Ranking (HITS and Page Rank)</td> <td>

https://pdfs.semanticscholar.org/43b6/d922bcfcc8f8fcd3e7d22c8dc732653d9571.pdf

 (video lessons available on drive for Google Group)
</td> </tr> <tr><th>2020</th> <td>Social Media Analytics A1</td> <td> </td> <td>

[[%ATTACHURL%/9a.SocialMediaAnalyticsA1.pdf][9a.SocialMediaAnalyticsA1.pdf]] [[%ATTACHURL%/9a.SocialMediaAnalyticsA1.pdf][<br />]]

[[%ATTACHURL%/9a.SocialMediaAnalyticsA1.zip][9a.SocialMediaAnalyticsA1.zip]]
</td> <td>Social Network Analysis PART A1: node-centric measures of influence</td> <td>

https://arxiv.org/pdf/1907.11229.pdf <span style="background-color: transparent;"> (a survey on trending topic detection from social networks )</span><br /> https://www.researchgate.net/publication/277689549_Efficient_temporal_mining_of_micro-blog_texts_and_its_application_to_event_discovery <span style="background-color: transparent;"> (SAX*)</span>

 (video lessons available on drive for Google Group)
</td> </tr> <tr><th>2020</th> <td>Social Media Analytics A2</td> <td> [[%ATTACHURL%/9b.SocialMediaAnalyticsA2.pptx][9b.SocialMediaAnalyticsA2.pptx]] [[%ATTACHURL%/9b.SocialMediaAnalyticsA2.pptx][<br />]]</td> <td> [[%ATTACHURL%/9b.SocialMediaAnalyticsA2.pdf][9b.SocialMediaAnalyticsA2.pdf]]</td> <td>Social Network Analysis PART A2: graph-based measures</td> <td>

[[%ATTACHURL%/Surveytopknodesinsocialnetworks.pdf][Surveytopknodesinsocialnetworks.pdf]]

 (video lessons available on drive for Google Group)
</td> </tr> <tr><th>2020</th> <td>

 Social Media Analytics B

 </td> <td style="text-align: left;"> [[%ATTACHURL%/10.SocialMediaAnalytics_community_detectionB.pptx.zip][10.SocialMediaAnalytics_community_detectionB.pptx.zip]] [[%ATTACHURL%/10.SocialMediaAnalytics_community_detectionB.pptx][<br />]]</td> <td> [[%ATTACHURL%/10.SocialMediaAnalytics_community_detectionB.pdf][10.SocialMediaAnalytics_community_detectionB.pdf]]</td> <td>Social Network Analysis PART B: community detection</td> <td>

 https://arxiv.org/ftp/arxiv/papers/1708/1708.00977.pdf

 https://hal.archives-ouvertes.fr/file/index/docid/804234/filename/Survey-on-Social-Community-Detection-V2.pdf

 (video lessons available on drive for Google Group)
</td> </tr> <tr><th>2020</th> <td>

 Social Media Analytics C

 </td> <td style="text-align: left;"> [[%ATTACHURL%/11.Maximizing-the-Spread-of-Influence.pptx][11.Maximizing-the-Spread-of-Influence.pptx]] [[%ATTACHURL%/SocialMediaAnalytics_community_detection_B.ppt][<br />]]</td> <td> [[%ATTACHURL%/11.Maximizing-the-Spread-of-Influence.pdf][11.Maximizing-the-Spread-of-Influence.pdf]]</td> <td>Spread of Influence in Social Networks</td> <td>

Chapter 2 of https://wiki.eecs.yorku.ca/course_archive/2014-15/F/4412/_media/social_networks.pdf

 (video lessons available on drive for Google Group)
</td> </tr> <tr><th>2020</th> <td>Opinion Mining</td> <td style="text-align: left;"> </td> <td> [[%ATTACHURL%/13.OpinionMining.pdf][13.OpinionMining.pdf]]</td> <td style="text-align: left;">Searching for opinions on the Web</td> <td>https://link.springer.com/content/pdf/10.1007/s10462-017-9599-6.pdf</td> </tr> <tr><th>2020</th> <td>Recommender Systems</td> <td style="text-align: left;"> [[%ATTACHURL%/12.RecommenderSystems.pptx][12.RecommenderSystems.pptx]] [[%ATTACHURL%/12.RecommenderSystems.pdf][<br />]]</td> <td> [[%ATTACHURL%/12.RecommenderSystems.pdf][12.RecommenderSystems.pdf]]</td> <td style="text-align: left;">Collaborative filtering, Content-based recommenders, Semantic recommenders</td> <td>

 http://shuaizhang.tech/2017/07/28/Summary-of-Recommender-System-Surveys-in-recent-years/ (a portal)

https://ieeexplore.ieee.org/document/8506344 ("A Survey of Collaborative Filtering-Based<br />Recommender Systems: From Traditional Methods to Hybrid Methods Based on Social Networks" 2018)

 (video lessons available on drive for Google Group)</td> </tr> <tr> <td> </td> <td>LABs</td> <td> *NOTE: UDATED LAB MATERIAL, INCLUDING VIDEOs IS SHARED THROUGH THE GOOGLE GROUP* </td> <td style="text-align: left;"> </td> <td>Lucene, Crawlers, Scrapers, Social Media, Temporal series</td> <td>(video labs available on drive for Google Group members)</td> </tr> <tr> <td colspan="6" rowspan="6"><br /> [[%ATTACHURL%/4Sax.pdf][<br />]] <br /> [[%ATTACHURL%/MavenCore.pdf][<br />]] <br /> [[%ATTACHURL%/TwitterApi.pdf][<br />]] <br /> [[%ATTACHURL%/5-Scraping.pdf][<br />]] <br /> [[%ATTACHURL%/CrawlCore.pdf][<br />]] <br /> [[%ATTACHURL%/TimeSeries_SAX.pdf][<br />]]</td> </tr> <tr> <td> </td> <td>Lab: Graph-G Library</td> <td> </td> <td> </td> <td> [[%ATTACHURL%/GraphLibraryStilo.pdf][Graph Libraries]]</td> <td> </td> </tr> </tbody> </table>

---++ Syllabus
---++ Part A: Web Information Retrieval
   * Introduction, Architecture of IR systems
   * Text processing, Indexing
   * Boolean and Vector Space Models
   * Query expansion, understanding users' needs
   * Ranking and query expansion based on word similarities (Singular value Decomposition, Word Embeddings)
   * Evaluation methods: experimental and theoretical methods
   * Web Information Retrieval
   * Link Analysis

---++ Part B:Social Information Extraction

   * Social Network Analysis: graph-based measures, community detection, topic diffusion, temporal analysis
   * Opinion Mining
   * Recommender Systems

---++ Topics for Final Dissertation

e-health, network medicine, event detection in social media and on the web, enterprise social networks, applications to social studies, temporal information retrieval, semantic recommenders, prediction and edetection of trendy topics

*Merit-based* (= high grades, very good programming skills) scholarships are available - ask the instructor-
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
zip	5.LSI.pdf.zip	r1	manage	8811.4 K	2020-03-13 - 15:49	PaolaVelardi
zip	5.LSI.pptx.zip	r1	manage	7130.3 K	2020-03-13 - 15:49	PaolaVelardi
pdf	7.link_analysis-compressed.pdf	r1	manage	1504.4 K	2020-03-31 - 10:42	PaolaVelardi
pptx	7.link_analysis.pptx	r5 r4 r3 r2 r1	manage	7732.8 K	2020-03-31 - 10:42	PaolaVelardi
This topic: Estrinfo > WebHome
Topic revision: r185 - 2023-10-17 - PaolaVelardi