Coding and Data Visualization

M 1-06

Welcome! Intro to the course, and Intro to XML and working in <oXygen/>: Write your first XML. Mindful File Management (how to name, store, share, and submit files in this course). Setting up accounts for web server and for GitHub.

Write your first XML with recipes:

• Slime recipe
• Bread recipe

• By Fri. 1/10: Install <oXygen/> software on your own computers. Instructions and license key posted on Canvas.
• Read our Introduction to XML, and read File-Naming Conventions for Canvas Homework Submissions.
• Complete XML Exercise 1
• Returning Students Only:
  • Choose one thing to respond to from last semester’s project evaluation. Respond in any way you wish: work on that project’s code, or turn this into a new and interesting coding challenge.
  • Meet with instructors to plan peer mentoring roles.

W 1-08

Getting started with GitHub and our DHClass-Hub. Discussion of the XML recipe homework: XML Comments and Well-formedness, and how to work with <oXygen/>. Introduce student projects from this class.

• Git Exercise 1: Read and consult our Guide to Git Shell as you complete this exercise.
• XML Exercise 2

F 1-10

Discussion of homework, and XML: Well-formedness vs. Validity. XML projects in digital humanities.

Hands-on GitHub: Working with our DHClass-Hub: Open and close Issues, write markdown in Issues "help tickets" and discussions, access our class examples of code. Command line: pull and push files in Troubleshooting directory.

• Git Exercise 2
• XML exercise 3: Review our feedback on your coding exercises so far and submit revisions if we asked you to. Mark up a text of your choice (any genre; manageable but reasonable size; foreign languages welcome). Work on applying attributes with your elements, and doing so in a careful and systematic way.
• Reading and Discussion Post on GitHub: Locate the class discussion exercise on our Issues board, which will ask you to read and discuss an article by Gabrielle Kirilloff, “<Traversing_the_Tree/>”, the short article, “Frankenstein novel analyzed” and scroll through Wendell Piez’s conference talk and images for the Balisage Markup Conference 2014. For full credit, your posts should make specific reference to passages in Kirilloff’s essay, and reflect on those passages. Please raise questions and respond to each other!

M 1-13

Overlapping hierarchies. Document Analysis and Coding: Group Exercise with Ozymandias

The concept of “UX.” or User Experience, and discussion of projects.

GitHub Practice (starting today for seven days): Make sure your personal repo is properly set up and cloned to your local computer. Using your Git Bash Shell, practice some basic Git commands to build a habit: Always pull before you push! Then push one file per day either to your personal repo or the DHClass-Hub. On alternating days, push to the other repo (if you pushed to your personal repo today, tomorrow you will push to the DHClass-Hub). Try cloning repos on multiple computers that you may be using through the semester (including computer lab machines on campus). Pull up our quick guide Using Git Through Command Line for a ready reference of commands and look up details in our Guide to Git Shell.

UX Discussion on DHClass-Hub Choose one of the following digital archives to explore, find the appropriate discussion thread on the DHClass-Hub, and write a post discussing: 1) how visual aids are used to convey information on the site, and how well (or badly) do these work? What seems unclear to you, and/or what seems to work really well? 2) how effective is the (“UX”) in navigating at least one or two significant areas of the site? (Respond to each other as the thread develops.)

• LOTR Project: Visualizing Tolkien’s Works on the Web
• What’s on the menu? (be sure to read the About page for the project goals)
• ORBIS: The Stanford Geospatial Network Model of the Roman World

Read Intro to Relax NG.

W 1-15

Schema Languages: Writing the Rules. Writing Relax NG

• Relax NG exercise 1: Write a Relax NG schema for one of the XML documents you created for an earlier assignment (XML exercise 2 or 3), and upload your schema and your XML file to Canvas.
• Review XML syntax and read and work with the Intro to Relax NG tutorial as you work on Relax NG Ex 1.
• Continue GitHub Practice (see M 1-14 assignment above).

F 1-17

Relax NG: mixed content, data types.

Quick tour of course projects with strong research questions developed by coding students at two Pitt campuses in the past few years.

Relax NG exercise 2: This time, choose a small text (maybe one of the letters from the first assignment that you did NOT mark up before, or anything you like). Perform document analysis, write a schema, and mark up the text according to the schema. Work with attributes, datatypes, and mixed content in your schema.

DHClass-Hub Discussion of UX and Project Research Questions for Pitt-Greensburg and Pitt-Oakland student projects

• The Hamilton Project
• The Ulysses Project
• The Banksy Project
• Profanities in Quentin Tarantino’s Screenplays (with network analysis)
• Divergences in Machine Translation: Examining Harry Potter texts processed by Google Translate across multiple language families
• Continue GitHub Practice (see M 1-13 assignment above).

M 1-20

Dr. Martin Luther King’s birthday observance: No classes.

W 1-22

Test 1 (in class: 15-20 minutes): XML markup.

Course projects, further discussion. Relax-NG issues.

Collect info for personal and project webspace on newtFire.

• Relax NG exercise 3: Choose a small text of a different type or genre than last time, perform document analysis, write a schema, and mark up the text according to the schema. Work with attributes, datatypes, and mixed content in your schema.
• DHClass-Hub Post and Discussion on Project Proposals:
  • Beginning from today through M 1-27: Post proposal ideas for team projects to work on this semester. Each student should post an idea for the class to consider: a project involving XML markup to be managed within a team of 2-4 students to investigate something we could study and visualize from our markup more effectively with computers than with human reading and description alone. All projects must involve a team of at least two persons, but this first exploratory proposal is an individual assignment.
  • Each student must respond to at least one of the proposed ideas from another student and indicate suggestions or further ideas. You may respond to more than one if you like, and indicate which proposals interest you to work on.
  • Proposal discussions will run until class time on M 1/28 when we will form project teams.
• Assignment for Dr. B: Set up and test student web space on NewtFire now.

F 1-24

Review Relax NG issues.

Coding for analysis: discussion of XML modeling for research questions. Network Analysis and Mapping with XML: Discussion of XML projects designed to explore networks: Tarantino, Banksy, Akira, Hamilton, Thalaba.

• Post a project proposal on DHClass-Hub if you have not done so already. Respond to another student's project proposal.
• Bad Graphs Reading and Discussion: Read A short tour of bad graphs and join our GitHub discussion exercise, which involves your posting examples of biased or otherwise problematic graphs you find on the web.

M 1-27

Form project teams.

Review bad graphs discussion. Ambitious graphs: See A Tour Through the Visualization Zoo.

Review Relax-NG: common issues in homeworks.

• Project Checkpoint 1: New project team leaders: initiate team GitHub repos and invite team members and instructors to join. Push content to your repo to initiate the project, such as a readme markdown file. Post and respond to the first issue(s) on your team repo, to (at least) schedule regular meeting times. Team members: respond to issues, help find a time to meet, and clone team repos.
• Review our comments on your Relax NG homeworks and practice writing Relax NG in preparation for the Relax NG test.

W 1-29

Project discussion / initiation time in class. Review of Relax NG issues, initiation of first take-home test (Relax NG).

• Test 2 (take-home): Relax NG
• Read our Intro to Regular Expressions and start looking at the Regular Expressions Quick Start in preparation for the next class.

F 1-31

Readme and other Markdown Files for your project GitHub

Introducing up-conversion with Regular Expressions:

• Using Find & Replace in <oXygen/>: hands-on exercise with Chocolate Chip Cookies recipe on DHClass-Hub and Regex Exercise 1: Shakespeare's Sonnets
• How to start: inside-out, or outside-in?
• The close-open strategy: The beginning of a thing comes at the end of the previous thing!

• Consult our Intro to Regular Expressions and the Regular Expressions Quick Start as you work on the exercise: Regex Exercise 1: Shakespeare's Sonnets
• FTP Client Setup: Follow the instructions at the top of this Syllabus page to install an FTP client of your choice on your computer, and follow our instructions posted on Canvas’s Announcements page to use that FTP client to access your web space on newtFire.

M 2-03

• Regex Exercise 2: a novel and speeches. Consult our Intro to Regular Expressions and the Regular Expressions Quick Start as you work on the regex exercises.

W 2-05

Regular Expressions: thinking algorithmically. Selecting for what’s not there. Simplifying overcomplicated expressions.

XML and the Web: Introduce HTML and CSS. SFTP into the Apache Server for newtFire. File directories and their association with web URLs.

• Project Checkpoint 2: Establish a clear file directory structure for GitHub, ensure all project team members are working consistently. Establish a project schema and associate that schema with project XML files in the repo. All team members work with and correct XML markup applying the project schema.
• Regex Exercise 3: Pacific Voyage Narrative
• Read our Introduction to XHTML (and HTML).

F 2-07

Building on the Web: HTML and CSS. Introduction to your web space on NewtFire and the index.html page

Hands-on: SFTP to NewtFire, association with GitHub.

Mindful file directory management: File directories and their association with web URLs.

• Consult our Introduction to XHTML (and HTML), and our Introduction to Cascading Stylesheets (CSS) as you work on HTML/CSS Exercise 1: NOTE: Do not submit this assignment on Canvas. Instead, submit this homework by using FTP to post to our newtFire web server, and do not follow standard homework naming conventions

M 2-10

HTML and CSS

• Web browsers and display variations
• CSS Box Model
• Controlling page layouts: Flexboxes tutorial, with simple examples.

• Consult Learn CSS Layout, Flexbox tutorial, and w3 Schools CSS Reference as you code. (Also, check out Paletton (or hunt for other color scheme generators on the web) to help think about choosing a balanced color scheme for your website. Experiment with writing CSS to control font, layout, color, backgrounds.
• HTML/CSS Exercise 2 (As with HTML Exercise 1, submit this homework by using FTP to post to our newtFire web server.)

W 2-12

HTML and CSS continued: Positioning and controlling layouts with HTML div, section, span, and CSS flex. Troubleshooting Server Side Includes

Review Regex issues in preparation for take-home test.

• Project Checkpoint 3:
  • Some team members work on developing page(s) for the project website, creating navigation menu (using a SSI), and working with section, div, and span elements. Try using CSS flex to control your horizontal layout. Work out a file directory structure for the GitHub repo to separate project website files from XML markup.
  • Others concentrate on completing markup, up-converting files to project XML with regex (as needed), and refining the project schema.

F 2-14

Project meeting time in class; completion of Project Checkpoint 3.

Review and take stock of technologies learned so far. Preview of what's next: XPath. Initiate Regular Expressions Take-Home Exam

• Test 3 (take-home): Regular Expressions
• Preview the Introduction to XPath in preparation for the next unit.

M 2-17

HTML5: Semantic web elements for accessible websites.

Introducing XPath and the XPath window in <oXygen/>: functions, axes, path steps /, and predicate filters [ ]. Hands-on: Understand XML as nodes and learn how XPath works, exploring Hamlet.

• First, carefully read our Introduction to XPath: Follow the XPath! As you read, try experimenting with the XPath expressions on our page, by downloading the explainXPath.html file, opening it in oXygen, and experimenting in the XPath window with some of our expressions. Then, move on to XPath Exercise 1.
• XPath Exercise 1

W 2-19

Using XPath axes to climb trees: child:: and descendant::, parent:: and ancestor::, preceding-sibling:: and following-sibling::, preceding:: and following::. The self:: axis, and when we need to refer to it in predicate expressions: [./node]. (Grouping) and [Position()].

• XPath Exercise 2
• Read and experiment with The XPath Functions We Use Most.
• Through next Monday: DHClass-Hub Discussion: Choose at least one of the following two posts by Steve Ramsay to continue reading and discussing on our DHClass-Hub:
  • The hermeneutics of screwing around, or
  • In praise of pattern
• Post at least twice before next Monday’s class, to reflect and respond to each other about DH and CS and one of the other two short articles by Stephen Ramsay on our DHClass-Hub Discussion of Stephen Ramsay’s work.

F 2-21

XPath Functions: some common functions we use and how to write them: count(), not(), distinct-values().

Preview functions with regular expressions in XPath.

• XPath Exercise 3: experimenting with functions
• Complete DHClass-Hub posts on Steve Ramsay Discussion.

M 2-24

Surgical precision with XPath on text strings: regular expressions in XPath functions

W 2-26

Introducing XQuery and the eXist-db on newtFire

Writing XPath in eXist-db

XPath String functions. XPath for the text() node to grab strings: string-join(), contains(string 1, string 2).

Hands on: learning to write XQuery: collections and documents, queries and FLWORs

• Read and check out some things you can try in our newtFire eXist environment at Michael Kay’s XQuery Tutorial and our Introduction to XQuery and the eXist XML Database.
• XQuery Exercise 1

F 2-28

• XQuery: Writing FLWOR statements and outputting HTML lists and tables
• Outputting files and saving them to the eXist-db database for previewing
• XQuery online and offline: in eXist and in <oXygen/>

• XQuery Exercise 2

M 3-02

• Review and troubleshoot XQuery Ex 2: FLWOR for-loops and outputs.
• Saving and Accessing files in the Newtfire eXist-db: set up individual and team project directories.
• Introduce Project Checkpoint 4: Markup Analysis Sprint

• XQuery Exercise 3: Upload one or more project files to the eXist-db database. Write XQuery to output HTML useful for your project, or work with the banksy directory. Save your query and, eventually, when your output is good, save the output to view from your personal directory following our model in 2019_ClassExamples. You must use a for-loop, and produce an HTML file containing a list and/or table of information.
• Prepare for Network Analysis: Install Cytoscape on your computer, together with the appropriate Java package. Note: This is also installed in the campus computer labs, so you may work with it there, too or instead of on your own computer. If you have trouble with the installation, locate Cytoscape in the campus computer labs. Begin familiarizing yourself with the Cytoscape interface, working with Cytoscape session files (with .cys extension) posted in Class Examples, Network-Workshop folder in our DHClass-Hub. Try importing one of my Cytoscape session files (.cys) following our tutorial instructions.
• Read An Introduction to Network Analysis and Cytoscape for XML Coders

W 3-04

XQuery from eXist to Web: Extracting from the XML database to publish in the Web Project

Network Analysis: Network graphs and structured data

XQuery to Network Analysis: generating plain text data (CSV or TSV) for Cytoscape

What we do in Cytoscape: Processing and Filtering; Running the Network Analyzer. Common network statistics: degree centrality, node centrality, edge betweenness, shortest paths

XQuery to Network Analysis: Exercise, Part 1

F 3-06

• XQuery to Network Analysis: Check for problems: TSV data extracted from XML.
• Importing to Cytoscape and running the Network Analyzer.

• XQuery to Network Analysis: Exercise, Part 2
• Project Checkpoint 4: Analysis Sprint: Complete as much of your project markup as you can, load good files into team project space in eXist-db, and experiment with extracting data, looking for patterns, visualizing networks (as applicable).

Sat. 3-07 - Sat. 03-14

M 3-16

Zoom meeting: Regrouping DH class in time of Covid-19 pandemic. Discussion: XQuery to Network Analysis: Survey issues and outputs.

Preview Schematron: Cleaning up project code with help from XPath

• Finish XQuery to Network Analysis: Exercise, Part 2. (Be prepared to discuss in our Zoom meeting on W 3/18 at 2pm.)
• Finish up remaining work on Project Sprint.

W 3-18

• Read our Guide to Schema Writing with Schematron.

F 3-20

Zoom class: writing an XPath-based Schema language: Schematron, why you need it now, and how we use it with Relax-NG

• Consult our Guide to Schema Writing with Schematron as you complete Schematron Exercise 1.
• Schematron Exercise 2: Write Schematron to refine and correct messy or problematic code for your project team. Submit two things: 1) your Schematron file, and 2) a sample of project XML code to show how you are capturing errors.
• Read and work through Fiona Carter’s Making a Basic Map with QGIS: In the process, download and install QGIS and complete the tutorial in preparation for our next class.

M 3-23

QGIS Exercise 1: XQuery to TSV for import to QGIS, and creating a map from project data.

W 3-25

QGIS Exercise 2: Map your data using QGIS

F 3-27

QGIS Exercise 3: Prepare a web map with Leaflet and post on your website.

M 3-30

Zoom class: XML that makes graphics: SVG (Scalable Vector Graphics). Drawing elements, and screen grid coordinates.

Introductory Slideshow. and w3Schools SVG Tutorial.

• Work through the w3schools SVG tutorial to complete the SVG basic and SVG shapes pages. Filters, gradients, examples, and reference are optional.
• SVG Exercise 1

W 4-01

Zoom class: XQuery to SVG: Pulling data for visualizing.

Plotting a timeline from project data: Part 1: Exploring the project data and doing “date arithmetic”.

• Before next class: Read and review what we discussed in class today on our tutorial for creating timeline infographic: global variables and user-defined functions. Look at how we're setting up XQuery to produce SVG.
• We will plot most of this SVG together in the Zoom class on Friday. You will finish it and prepare the data for visualizing in your own way for the homework assignment due after that class: (Friday 4/3 by 11:59pm).

F 4-03

Zoom class: Work together to plot an SVG timeline from our date-arithmetic work on the Rocket Launches project.

Making line and bar graphs, plotting axes. How to plot stacked bars.

M 4-06

Zoom class: Example: Putting the pieces together: SVG and HTML. Introducing JavaScript, the DOM, JavaScript event listeners and functions.

W 4-08

• Zoom class: Working with global variables and for loops in JavaScript. JavaScript with CSS over SVG
• Project Visualization and Mapping needs: discussion

F 4-10

Zoom class: JavaScript? or More SVG/Visualization

Project Development, prepare presentation material.

M 4-13

Zoom class: Projects and Portfolios: Guidelines for Submission. Creative Commons Licenses

Web development for Project Checkpoint 5

W 4-15

F 4-17

Thurs. 4/23 by 11:59 PM

Projects due. Submit projects with a post to the instructors from your project GitHub repo.