Crowdsourcing is a form of human computation.in which people undertake tasks that we might consider assigning to a computing device, e.g. a language translation task. A crowdsourcing system enlists a crowd of humans to help solve a problem. The availability of crowdsourcing services is now making human computation easily available to the research community. There is currently significant interest in the use of crowdsourcing services to support existing research activities in information and data processing technologies; Interest in crowdsourcing information retrieval has largely focussed on its use for relevance assessment in the creation of test collections. However, it is also being explored for tasks such as the creation of search queries for test collections, and more generally to understand potential human information needs. This presentation will introduce the topic of crowdsourcing including the issues of recruitment, management and payment of crowdsource workers, and the design of tasks which can be assigned to workers to support information retrieval research.

Provides an introduction to classical IR evaluation methods, along with an overview of more recent approaches. Our primary focus is on offline evaluation through the construction of reusable test collections, but we briefly cover online evaluation, user studies and related topics. We draw examples from TREC, NTCIR and other evaluation efforts. We take a user-oriented approach to evaluation metrics, including a review of traditional metrics from this perspective.

We then discuss sources of user behavior that can be used to develop and calibrate user models for IR evaluation, including lab studies and implicit feedback. Finally, we introduce techniques for developing evaluation methodologies that directly model key aspects of the user's interaction with the system. The overall goal of the lecture is to equip researchers with an understanding of modern approaches to IR evaluation, facilitating new research on this topic and improving evaluation methodology for emerging areas.

Information Retrieval (IR) is the study of finding relevant information in unstructured collections of material using nothing more than a poorly specified query. In this lecture, I will detail the development of IR research: from its electro-mechanical origins in the early 20th century through early approaches to ranking before describing the broad range of techniques used by modern search engines today. The techniques I will introduce include text processing (e.g. stop word removal, stemming, different languages), weighting methods (e.g. tf*idf, length normalisation, BM25), link analysis (e.g. PageRank), retrieval models (e.g. vector space, probabilistic, learning to rank), and extraction of information and user context from interaction logs. The means by which search engines can be evaluated will also be outlined.

As this is only a relatively short introductory lecture, the aim of the presentation is to ensure that the student understands the broad range of research methods that are used to try to overcome the challenge of IR. If time allows, some description of future trends in IR will also be presented.

Information retrieval is an area in computer science about technologies for helping people to access information and knowledge. Machine learning plays an important role in modern information retrieval. In this lecture, I will introduce three groups of machine learning technologies for information retrieval, namely `learning to rank’, `learning to match’, and `deep learning for information retrieval’. I will mainly explain the key points of the technologies through introduction to the methods which I and my former and current colleagues have developed. For learning to rank, I will describe the three basic approaches including pointwise, pairwise, and listwise approaches. For learning to match, I will indicate that semantic matching is the key issue for search, and machine learning can help solve the problem. For deep learning for information retrieval, I will argue that deep learning is particularly useful for hard problems in information retrieval, including question answering from knowledge base, image retrieval, and generation based question answering.

Digital information is increasingly available in multimedia and multimodal forms incorporating various combinations of audio-visual media, including images, video, speech, music, and textual metadata. Information retrieval for multimedia content poses many challenges beyond those of text retrieval. These include the need to analyse discover the content to enable the material to be indexing for searching, and the requirement to develop interactive user interfaces to enable effective navigation and audition of retrieved content. The presentation will introduce the key issues of multimedia information retrieval including recognition and indexing of audio and visual content, exploitation of textual metadata in multimedia information retrieval, and design of user interfaces for interaction in multimedia information retrieval.

Provides an introduction to core concepts in the online evaluation of Information Retrieval system. The first segment of the lecture will focus on A/B testing for search, including query stream partitioning, statistical testing, and interpretation of results. The second segment will cover a variety of inter-related topics, including practical issues of logging, counterfactual evaluation, result randomization, interpretation of clicks and other user actions. Our goal is to provide a board overview of these topics, equipping researchers with a basic toolset for work in this area.

Traditionally, information retrieval (IR) is considered an individual pursuit. Not surprisingly, the majority of tools, techniques, and models developed for addressing information need, retrieval, and usage have focused on single users. The assumption of information seekers being independent and IR problem being individual has been challenged often in the recent past. This lecture will introduce such works to the students, with an emphasis on understanding models and systems that support social and collaborative search (SCS). To put SCS in perspective, the course will show the students how various related concepts, such as collaborative information seeking, social information seeking, collaborative filtering, and social search are related and differentiated. This contextualization can then be useful for exploration of concepts and development of tools, as well as for evaluation purposes.

Specifically, the course will (1) outline the research and latest developments in the field of SCS; (2) list the challenges for designing and evaluating SCS systems; and (3) show how traditional single user IR models and systems could be mapped to those for SCS. This will be achieved through introduction to appropriate literature, algorithms and interfaces that facilitate social and/or collaborative search, and methodologies for studying and evaluating them. Thus, the lecture will offer a balance between theoretical and practical elements of SCS.

In essence, the lecture will introduce the student to theories, methodologies, and tools that focus on information retrieval/seeking in social and/or collaborative settings.

Mining social media and social networks are important research topics due to the increasing presence of users' online personas, online social links and online activities. These user generated social media data represent a wealth of information for urban, social and business applications. Beyond helping users to interact with one another and to seek information, researchers now analyse social media data to discover knowledge of user profiles, opinions, behavioural patterns, and social networks. Social media analytics research for social and consumer insights and for turning insights into useful applications are challenging. In this tutorial, we shall first describe the challenges and present a social media analytics framework. We will then focus on user profiling and behaviour mining applications and the relevant information retrieval techniques. Finally, we show a few example social media analytics applications that demonstrate the integration of information retrieval techniques to address some urban and social needs.

When users interact with Web search engines, they leave rich implicit feedback information in the form of submitting queries, query reformulations, result clicks, cursor movements etc. These user behavior signals contain valuable information about how users seek information in Web search scenarios and therefore are extremely important for the understanding of the information perceiving process of human and performance improvement of search engine systems. In this lecture, I will start by talking about the behavior patterns of users in querying, examining and clicking results during both desktop and mobile search processes. Based on these existing findings in user search behavior, we will go through some popular behavior models which bring us from single user to user crowds, from homogeneous to heterogeneous result environments and from single-query to multiple-query sessions. After that, we will use some examples to show how these models help search engines to improve the performance of result ranking, user intent understanding and user interface/evaluation metric designing.