NK CS349 Blog

Milestone 3

2009-12-10T18:50:00.000-08:00

For Milestone 3, we redesigned the interface. We thought it was too involved, confusing, not-intuitive enough. So, we make the interaction screen smaller, and gave a Rehearsal obejct 3 main items: timeline, movie, and notes panel.

Other items were also added to the application: a Recent Notes panel, a seeker for the movie (to navigate to certain times), a timestamp for the seeker, a title bar, a panel for buttons to filter the notes by color and type, and a portion that displays who is logged in. These objects improve the functionality and usefulness of the application.

The portions that I personally worked on was getting the movie to pause and play via tapping and to get the seeker to work. Both were a matter of looking at documentation for the movie object which is an instance of MediaElement. this object has built in functions for playing and pausing -- the playing and pausing of the movie is achieved via making Play and Pause buttons that are as large as the movie screen and invisible; so, when the user taps the screen, an invisible button is pressed with the correct functionality.

The MediaElement documentation gave a method for making s lider that could be used to slide to a certain portion of the video. However, in action, it is very slow/inaccurate/effectively stops the movie when used. i.e. interaction is very inconsistent and its is not very useful.

What we actually wanted was the idea of a "play head", that moved along with the video so that users could see where they were along their timeline, to give them more insight into their notes. However, to get this object to dynamically update and somehow be linked with the position of the movie element proved to be a problem out of the scope of the project/time/my knowledge.

In the end, I feel as though this prototype is a lot more approachable than the last one. My biggest concerns with it lie in the fact that I think the interface is still very GUI-like and not very intuitive.
I do believe that our project is more tangible than it is appears, as we did not get to implement the RFID, video camera, and pen portions, which were pivotal to the idea of our system integrating into normal rehearsal behavior.

Milestone 1

2009-12-01T12:26:00.000-08:00

In preparation for Milestone 1, a lot of effort was put into understanding how to code for the Microsoft Surface and how to get Surface objects to interact with code written for functionality.

The main task I worked on was trying to get the Timeline image to generate, as this is a pivotal part of our interface and helps to personalize a user's experience.

It became apparent before working on generating the Timeline image, that it would be necessary to clarify what sorts of objects would be necessary to create (e.g. Movie, Note, Person, etc.), and to get an idea of what functionality belonged to each class. So, the first step was to conceptualize the necessary objects and to make skeletons of their classes.

After this, came the struggle of figuring out how to generate and appropriate image and associate that with a certain object on the Surface's screen. Pointers were taken from the GeneBrowse code, as it is a great resource (albeit, hard to read in its entirety).

The task of generating the Timeline was broken into: generating the black background rectangle, generating the rectangles that represent the duration of a person's time on the screen, and to generating note markers of the appropriate type in the appropriate place. Of course, the stage-time rectangles and the note-markers had to be generated according to a color associated with the particular Person that the stage times referred to.

These three tasks were eventually implemented and generated (based on canned information).

The second big task that was approached this milestone (and implemented by Donna), was taking in text notes taken during production and parsing them properly so that they're associated with the correct person and generate the correct type note-marker at the correct timestamp on the Timeline.

The layout design was refined by Rebecca and Ji -- but upon seeing everything come together we all thought that the interface looked very cluttered and not very intuitive (and the class agreed). We also agreed that our design does not leverage the Microsoft Surface's inviting qualities or collaborative qualities.

So, for the next milestone, a lot of things need to transpire, including a major overhaul of the interface.
Other tasks we have set out (in no particular order):

- experiment with the LogiPen
- work on getting the parsed notes to generate a Notes panel
- enable Notes to be dragged out into a separate manipulable object
- be able to tap the Timeline and have this bring up the Notes re-sorted menu
- get the the Notes re-sorter menu to re-sort the Notes according to type and/or color
- get the Movie to pause and play via tapping
- implement a Recent Notes panel
- implement a Tape Head that moves along the Timeline (and is also draggable for fast forwarding/rewinding)

Project Dynamic Rehearsal

2009-11-10T12:17:00.000-08:00

Our design concept was to create a tool to make reviewing rehearsal more effective and collaborative. To this end, we designed a recording system that would be unobtrusive during rehearsal as well as a Microsoft Surface applicationto be used for review.

During rehearsal, a video camera will be capturing video and audio. Whomever customarily takes notes during rehearsal -- e.g. director, line reviewer, tech director -- takes notes as they normally do. Their note-taking process is augmented by the fact that they are using a digital pen and transfers their notes to a digital format as well as records the time stamp of when the note was taken.
Actors all wear RFID tags and enter the stage through an entrance that is augmented with an RFID detector. Their presence on stage is thus automatically recorded.

The timestamped notes and the times of when the actors on stage are paired with the recorded video via the Microsoft Surface. The videos is automatically annotated with notes and these notes are automatically attributed to the actors, if the timestamp of the note corresponds with their time on stage.

After the rehearsal, actors can log into the annotated performance by placing their hand or RFID tag on the surface. They are automatically presented with a workspace of their most recent rehearsal. This workspace includes their most recent unread notes as well as: the video of the rehearsal, a small timeline that graphically depicts when they were on stage and where on the timeline notes were taken, an area where a portion of the small timeline can be zoomed into. Notes from different people are indicated by the shape of the note indicator on the timeline.

Multiple users (up to 4) can log into the program to get a workspace that includes timelines that display all of their performances as well as notes that are pertinent to 2 or more of them. Users are differentiated on the timeline and the notes by color.

Exploring Tern through TUI Frameworks

2009-10-08T09:57:00.000-07:00

Tern and Reality-Based Interaction Themes

Interaction with Tern relies on the user's innate Naive Physics understanding and Environmental Awareness Skills. User's understand that, to use the system, they need to pick up puzzle pieces and place them on a surface. They also understand that puzzle pieces fit together -- i.e. they cannot occupy the same spot and should not be placed on top of one another. User's also understand that their program will not be complete until the puzzle is complete.
User's understand that to create something, they must manipulate the pieces (BAS). Also, they understand that changing an element of Tern will affect the outcome of the program.
Social Awareness Skills are imployed as teams collaborate to create a cohesive program. It is natural for user's to gather around a Tern program and to relate to one another. Also, the fact that multiple teams can display their results at the same time employes SAS.
The beauty of Tern is that it leverages the natural understanding of students to put puzzle pieces togehter (NP) and to collaborate in create something functional/piece of art (SAS). The manipulation of the pieces comes naturally because of their shape (BAS).

Tern Specification Using the TAC Paradigm

TAC #	Token	Constraint	Variable	Action	Observed Feedback
1	Puzzle Piece	Puzzle Pieces Work Surface Scanner	Puzzle Piece	Add/Remove	Add/Remove puzzle piece from the work surface Add/Remove command from the program
2	Scanner	Scanner	Scanner	Utilize	Scan and compile program

Tern - Tangible Programming Language for Young Minds

2009-09-22T17:39:00.000-07:00

I offer my reflections after I completed reading a paper on Tern and presenting a PowerPoint.

I enjoyed the concept for the tangible programming language, as I have a personal interest in programming. When I first understood that Tern was about converting puzzle-piece code into compiled programs, my mind jumped to the idea that the puzzle pieces themselves had embedded electronics (I think I got this idea from the Tangible Movie Editor). However, I love the fact that the puzzle pieces are actually wooden and contain nothing delicate -- obviously this is a great option for children. I also like that the images on the blocks and their SpotCodes can be printed out and glued onto the inexpensive blocks if blocks get lost.

One thing I'm curious about is the value of having pre-schoolers and elementary-schoolers write programs. I understand that Tern would be a very gentle introduction to programming; but, I feel like it's almost too easy and the children may not be able to apply the very basic ideas of Tern to actual programming later on. Could Tern effectively teach children more advanced topics -- e.g. recursion, object-oriented language, parameters. Could children translate their understand of Tern into these harder concepts? Is it practical to teach pre-schoolers how to program? I imagine that these questions would have to be answered by longer-term studies.

Cell Tango

2009-09-21T10:21:00.001-07:00

Cell Tango could be enhanced via aspects of TUI design. Obviously, Cell Tango is not a Tangible User Interface -- most simply because it is not tangible. It could be augmented in many ways, however, to be made tangible.

perhaps, there could be a tangible way of users adding photos to the project in real-time to see how that affected the webs of words and the various animations. perhaps adding a scanning station for photographs, or placing a photo on the screen and having that transpose into the cell tango.

perhaps the images on the screen could be manipulated by users. this could be done in a way similar to the Microsoft surface. users could move images around by touching/flicking them, to discover how animations would rearrange themselves.

perhaps the words on the screen could be manipulated by users, as well. this would be interesting, as users could see how the word webs change as more words are added or as words are taken away.

perhaps users could add just words in real-time, and see what the system grabs images from the database as well as from Flickr.

a benefit of making Cell Tango a tangible interface would be enhancing the idea that Cell Tango is to be customized to a community. it could take that one step farther, allowing an individual or individuals to create their very own personal Cell Tango.