Advanced Topics in Human-Computer Interaction

(14th July) The submission due for your demonstration video is 28th 23:59 JST. Please find more details about how to submit here.

(9th April) I have made paper assignments for those who told me their preferences. Check the reading list and find your name. Starting reading the assigned paper and prepare for your presentation. If you don't find your name, please send me an email.

(8th April) For those who are taking this course, please name your preferences on paper reading as soon as possible (by 9th April, strongly preferred). The link is here.

My course of Advanced Topics in HCI includes discussions about a set of representative papers published in the field of HCI, and creation and demonstration of interactive systems. Students lead their own capstone projects where they build interactive systems and present their demonstrations at the last class.

This course is double-listed as “3747-108: Advanced Topics in HCI” in Graduate School of Engineering and “4915100: Human Interfaces” in Interfaculty Initiative in Information Studies, Graduate School of Interdisciplinary Information Studies. Students are allowed to register only to either of these two courses. Everything besides a course name is the same, so no worries about which one you should register. :)

この講義は工学系では「3747-108: ヒューマンコンピュータインタラクション特論」，情報学環・学際情報学府では「4915100: ヒューマンインタフェース」として提供されています．学生はどちらかの講義にしか登録できません．講義の名前以外はすべて同じですので，どちらかで登録していただければ結構です．:)

This course has two major objectives: getting familiarized with classic and recent HCI research that well demonstrates novel interactive systems and applications, and designing an interactive system by conducting brainstorming, qualitative surveys, low-fidelity prototyping, and working prototype implementation. To achieve them, this course offers a mixture of research discussions on HCI papers and capstone projects.

• Research discussions: We discuss a selected set of papers published at HCI and its relevant conferences, such as CHI, UIST, UbiComp, CSCW, and MobiSys. Each student will be asked to lead discussions at least once during the semester.
• Capstone projects: Students conduct a project to build an interactive system. They will be asked to do a demonstration of their systems at the end of the course.

English is the official language in this course though Japanese may be used if necessary. All teaching is done in English at a class. Students are strongly recommended to deliver their presentations and demonstrations in English. You may use Japanese when you have large difficulties in communication, but you must always try your best to speak English.

We do not have any explicit prerequisite for this course, but students are expected to have:

• Basic knowledge and experience on HCI research,
• Programming skills and experience, and
• English communication skills.

But, the most important is, of course, your strong passion. :)

We have no tolerance to any type of academic misconducts, such as plagiarism, inappropriate citations, and fabrications. Examples are:

• Using codes and/or libraries without citing appropriately,
• Using source codes written by others without explicit permissions，and
• Making up data or system behavior for better-looking demonstration.

In case serious academic misconducts are found, we give following strong penalties depending on their significance.

• No mark for assignments where academic misconducts are found (Marked as zero. Marks are retracted if already given),
• No mark for all assignments that have been already submitted,
• No mark for all assignments that have been already submitted and prohibition to submit future assignments.

Please make sure that your reports and source codes do not cause misunderstandings.

Auditing students are welcome to join us. But I strongly recommend you to do a discussion chair even if you are just auditing. Also participate in discussions at the class. Just don't be a free rider. :)

Your performance in this course will be evaluated in the following criteria.

• [25%] Paper discussion: Given to your performance in leading discussions about the paper assigned to you from the reading list.
• [60%] Capstone project: Given to the quality of your qualitative study and system demonstration.
  • [15%] Project progress: Given to the management and progress of your project.
  • [45%] Project demonstration: Given to the originality, technical novelty, and thoroughness of your system demonstration.
• [15%] Engagement and attendance: Given to your attendance to the course and your involvement in discussions during the class.

You must both do a discussion chair at least once and complete your capstone project to get a final mark. Otherwise, your mark will be zero.

We mainly discuss the papers written in bold during classes. All students must read them before coming to the class. If you are strongly interested in any of these topics, I recommend you to read optional readings. At least, you should watch their videos. :)

• 14th, April: Sensing touch
  • DiamondTouch: a multi-user touch technology in UIST 2001. (paper) (video)
  • SmartSkin: an infrastructure for freehand manipulation on interactive surfaces in CHI 2002. (paper) (video)
  • [Jan Meier] Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects in CHI 2012. (paper) (video)
  • Optional readings
    • A research center for augmenting human intellect in 1968. (paper)
    • The mother of all demos in 1968. (link) (video)
    • A multi-touch three dimensional touch-sensitive tablet in CHI 1985. (paper) (video)
    • Low-cost multi-touch sensing through frustrated total internal reflection in UIST 2005. (paper) (video)
    • PlayAnywhere: a compact interactive tabletop projection-vision system in UIST 2005. (paper) (video)

• 21st, April: With my hand, on my body
  • [Becky Su] Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces in CHI 2008. (paper) (video))
  • Skinput: appropriating the body as an input surface in CHI 2010. (paper) (video)
  • OmniTouch: wearable multitouch interaction everywhere in UIST 2011. (paper) (video)
  • Optional readings
    • Twiddler typing: One-handed chording text entry for mobile phones in CHI 2004. (paper)
    • Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments in CHI 2000. (paper)
    • “FingeRing”: a full-time wearable interface in CHI 1994. (paper)
    • GestureWrist and GesturePad: unobtrusive wearable interaction devices in ISWC 2001. (paper)
    • The gesture pendant: A self-illuminating, wearable, infrared computer vision system for home automation control and medical monitoring in ISWC 2000. (paper) (video)
    • Epidermal electronics in Science 2011. (paper)

• 28th, April: Crazy visions
  • [Mario Rodriguez] KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera in UIST 2011. (paper) (video)
  • HeatWave: thermal imaging for surface user interaction in CHI 2011. (paper) (video)
  • HyperCam: hyperspectral imaging for ubiquitous computing applications in UbiComp 2015. (paper) (video)
  • Optional readings
    • Eulerian Video Magnification for Revealing Subtle Changes in the World in SIGGRAPH 2012. (paper) (video)
    • Femto-photography: capturing and visualizing the propagation of light in SIGGRAPH 2013. (paper) (video)

• 12th, May: Gestures captured
  • Put-that-there: Voice and gesture at the graphics interface in SIGGRAPH 1980. (paper) (video)
  • [Huang Yifei]　TouchLight: an imaging touch screen and display for gesture-based interaction in ICMI 2004. (paper) (video)
  • Optional readings
    • A hand gesture interface device in CHI 1987. (paper)
    • A survey of glove-based input in IEEE Computer Graphics and Applications 1994. (paper)
    • Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor in UIST 2012. (paper) (video)
    • Real-Time Human Pose Recognition in Parts from a Single Depth Image in CVPR 2011. (paper) (video)

• 19th, May: Turning reality to surreality
  • A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment in Personal Technologies 1997. (paper) (video lecture by Steve Feiner)
  • Augmented surfaces: a spatially continuous work space for hybrid computing environments in CHI 1999. (paper) (video)
  • [Tung Ta D.] IllumiRoom: peripheral projected illusions for interactive experiences in CHI 2013. (paper) (video)
  • Optional readings
    • A taxonomy of mixed reality visual displays in IEICE TRANSACTIONS 1994. (paper)
    • Interacting with paper on the DigitalDesk in Communication of ACM 1993. (paper) (video)
    • Annotating the Real World with Knowledge-Based Graphics on a See-Through Head-Mounted Display in GI 1992. (paper)
    • The world through the computer: computer augmented interaction with real world environments in UIST 1995. (paper) (video)
    • HoloDesk: direct 3d interactions with a situated see-through display in UIST 2012. (paper) (video)

• 26th, May: Feeling your force
  • [Takuma Yoshitani] TeslaTouch: electrovibration for touch surfaces in UIST 2010. (paper) (video)
  • PossessedHand: techniques for controlling human hands using electrical muscles stimuli in CHI 2011. (paper) (video)
  • [Koya Narumi] Affordance++: Allowing Objects to Communicate Dynamic Use in CHI 2015. (paper) (video)
  • Optional readings
    • Active click: tactile feedback for touch panels in CHI EA 2001. (paper)
    • Ambient touch: designing tactile interfaces for handheld devices in UIST 2002. (paper)
    • Tactons: structured tactile messages for non-visual information display in AUIC 2004. (paper)
    • Tactile brush: drawing on skin with a tactile grid display in CHI 2011. (paper)
    • Mudpad: tactile feedback and haptic texture overlay for touch surfaces in ITS 2010. (paper) (video)
    • Noncontact Tactile Display Based on Radiation Pressure of Airborne Ultrasound in IEEE Transactions on Haptics 2010. (paper (video)
    • Multimodal collaborative handwriting training for visually-impaired people in CHI 2008. (paper)

• 9th, June: Tangibles and beyond
  • [Tianyu YUAN] Tangible bits: towards seamless interfaces between people, bits and atoms in CHI 1997. (paper) (metaDesk video)
  • [Ken Tominaga] inFORM: dynamic physical affordances and constraints through shape and object actuation in UIST 2013. (paper) (video)
  • Optional readings
    • Bricks: laying the foundations for graspable user interfaces in CHI 1995. (paper) (video)
    • The metaDESK: models and prototypes for tangible user interfaces in UIST 1997. (paper) (video)
    • Urp: a luminous-tangible workbench for urban planning and design in CHI 1999. (paper) (video)
    • Comparing the use of tangible and graphical programming languages for informal science education in CHI 2009. (paper) (video)
    • Topobo: a constructive assembly system with kinetic memory in CHI 2004. (paper) (video)
    • Tangible interfaces for remote collaboration and communication in CHI 1998. (paper) (video)
    • RobotPHONE: RUI for interpersonal communication in CHI EA 2001. (paper) (video)
    • Phidgets: easy development of physical interfaces through physical widgets in UIST 2001. (paper)

• 16th, June: Super DIY
  • [Thien Che] Interactive construction: interactive fabrication of functional mechanical devices in UIST 2012. (paper) (video)
  • [Takefumi Hiraki] Foldio: Digital Fabrication of Interactive and Shape-Changing Objects With Foldable Printed Electronics in UIST 2015. (paper) (video)
  • [Danielle DeLatte] Protopiper: Physically Sketching Room-Sized Objects at Actual Scale in UIST 2015. (paper) (video)
  • Optional readings
    • Printed optics: 3D printing of embedded optical elements for interactive devices in UIST 2012. (paper) (video)
    • The FreeD: a handheld digital milling device for craft and fabrication in UIST EA 2012. (paper) (video)
    • Printing teddy bears: a technique for 3D printing of soft interactive objects in CHI 2014. (paper) (video)
    • LaserOrigami: laser-cutting 3D objects in CHI 2013. (paper) (video)
    • faBrickation: fast 3D printing of functional objects by integrating construction kit building blocks in CHI 2014. (paper) (video)

• 23rd, June: UIST 2015 highlights (Reading is optional.)
  • Patching Physical Objects. (paper) (video)
  • Encore: 3D Printed Augmentation of Everyday Objects with Printed-Over, Affixed and Interlocked Attachments. (paper) (video)
  • Orbits: Gaze Interaction for Smart Watches using Smooth Pursuit Eye Movements. (paper) (video)
  • GelTouch: Localized Tactile Feedback Through Thin, Programmable Gel. (paper) (video)
  • SensorTape: Modular and Programmable 3D-Aware Dense Sensor Network on a Tape. (paper) (video)
  • TurkDeck: Physical Virtual Reality Based on People. (paper) (video)

• 7th, July: CHI 2016 highlights (Reading is optional.)
  • FlexCase - Enhancing Mobile Interaction with a Flexible Sensing and Display Cover. (paper) (video)
  • TableHop: An Actuated Fabric Display Using Transparent Electrodes. (paper) (video)
  • SkullConduct: Biometric User Identification on Eyewear Computers Using Bone Conduction Through the Skull. (paper)
  • Motion Guidance Sleeve: Guiding the Forearm Rotation through External Artificial Muscles. (paper) (video)
  • Body Integrated Programmable Joints Interface. (paper) (video)
  • ExoSkin: On-Body Fabrication. (paper) (video)
  • HotFlex: Post-print Customization of 3D Prints Using Embedded State Change. (paper) (video)

Though we will not go through this list during the class, if you are into this space, I recommend you to read these papers. :)

• Activity Sensing
  • Activity Recognition from User-Annotated Acceleration Data (Pervasive 2004)
  • StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones (UbiComp 2014)

• Mobile Interaction
  • Sensing techniques for mobile interaction (UIST 2000)
  • Ambient touch: designing tactile interfaces for handheld devices (UIST 2002)
  • SideSight: multi-"touch" interaction around small devices (UIST 2008)
  • Acoustruments: Passive, Acoustically-Driven, Interactive Controls for Handheld Devices (CHI 2015)

• Sensing for health
  • SpiroSmart: using a microphone to measure lung function on a mobile phone (UbiComp 2012)
  • Augmenting gesture recognition with erlang-cox models to identify neurological disorders in premature babies (UbiComp 2012)

In research discussions, we discuss some of recently-published HCI work that demonstrates strong novelty and/or progress in this field. After the first class, please name your preferences in this page.

• Discussion chair: This person plays a central role of stimulating discussions among fellow students. You will have 25 minutes in total for your discussion slot. You must read the assigned paper carefully, and deliver a 10-minute presentation. After your presentation, you will be expected to lead discussions with fellow students. Your presentation material must be in English though you can deliver either in English or Japanese. Your presentation should cover:
  • Backgrounds of the research,
  • Summary of the developed system,
  • Novelty and originality of the work, and
  • pros and cons of the system/method.

• Discussion members: The rest of you will serve as discussion members. You must engage in discussions proactively. All of discussion members must read the papers before coming to the class. You should take notes about your impression on the papers, in particular:
  • What did you like in this work? Why?
  • How do you think this work can inspire your research?
  • What are possible applications out of this technology?
  • What are shortcomings? What improvements do you think this technology needs?
  • If you were a reviewer on this paper, how would you rate and provide feedback?

In Class #1 , we discuss some vision videos. If you are interested in checking more videos, use the following links to find your favorites.

• Vision videos
• HCI vision videos

A capstone project aims to obtain experience of building an interactive system with hardware and delivering a live demonstration.

You are encouraged to collaborate with your fellow students and team up for capstone projects. However, your team must be up to three people. I recommend to work in a team of two. Marks for the capstone project will be given equally to all team members.

Your system must be interactive and use some kinds of hardware. You may use anything for your project. Your hardware can be smartphones, cameras, Kinect sensors, and/or what you build by yourselves. As one of the objective of this assignment is to experience hardware hacks, projects with codes only are not acceptable.

You will be asked to do a live demonstration at the last class. So make sure that your final system works in real time. Your system will likely to perform some sort of recognition (heuristically or with machine learning). The recognition does not have to be super accurate, but it has to work reasonably well.

We do not care about what programming languages or environment you use. If you need suggestions or support, please consult with the instructor though we do not guarantee providing the stuff you want.

You must deliver the following items at the end of the course.

• Project presentation and live demonstration: Roughly 10 mins presentation. It must include a live demonstration of your system.
• Demonstration video: A video that shows a demonstration of your system. The video should be under 5 minutes long and 100MB. MP4 is recommended, but a common video format (e.g., mpeg, avi, wmv, and mov) is also acceptable.

We evaluate your capstone projects in the following criteria:

• [10%] Originality: The uniqueness of your system in terms of the concept, design, and/or implementation.
• [15%] Implementation thoroughness: The quality of your implementation. This includes the complexity, scalability, and technical difficulty of your system.
• [10%] Interaction design: The usability and quality of the designed interaction.
• [10%] Presentation delivery: The quality of your presentation (including your live demo).

Examples of capstone projects are as follows (but not limited to):

• Recognizing user's activities from sensor data on a smartphone
• Detecting gesture input to support a new type of interaction with computers
• Detecting user's different types of physical exercise
• Creating new visual environments for entertainment
• Enabling concepts that are shown in the vision videos we discussed at the first class

After the last class, each team will have another two weeks to refine the system and shoot a demonstration video. The demo video should describe the walkthrough of your system and some details about the implementation. I expect each video to be up to five minutes and under 100MB though these are not a hard limit.

Before creating your demo video, please review those published in CHI, UIST, and/or other major HCI conferences. The videos we have seen during the class would be good to review too. These videos will give you a better idea of what you should have in your submission.

The due date for your video submission is 28th July, 23:59 JST. Please upload your video to major cloud services (e.g., dropbox, onedrive, google docs), and send its link to me by email. In addition, please describe the following information in the body of that email.

• Information about all team members
  • Name (in English)
  • Student ID
  • Email address
• Your preference on publishing demo videos
  • If you are ok with publishing, I will upload it to YouTube, and future students can watch for their inspiration for capstone projects.
  • If you are not ok with publishing, please let me know.
• Any opinion and comment on this course
  • Suggestions for next year would be greatly appreciated!
  • I promise that any of your opinions and comments never affects your performance negatively.

If you have any trouble or question on submitting your video, please contact the instructor well in advance.