Trials of the Datamancer

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br><br>(align: "=><=")[ <span style="padding-top:80px;"><h1>Trials of the </font><font color="#C12283"><i><b>Datamancer</i></b></font></h1>]</span> <div class="introcontainer"> (align: "=><=")[The way in which data is interpreted often relies on how the data is represented, or visualized. For example, you will interpret data differently in a spreadsheet than in a bar graph, or a map. Different kinds of data visualizations allow for a broader interpretation of the data, and lead to deeper analysis. This is a tutorial for how to create map, with CartoDB, from a dataset. In this tutorial, you will also learn how to clean, scrape, and organize your data using tools such as Import.io and regular expressions. ### Begin your education Learning about new tools can also be fun. The authors of this tutorial have structured it like a game. To begin the tutorial, click on one of the paths below. <b>If you want to start with a clean dataset, then </b>[[your journey begins at dawn...]] <b>If you want to start from without a dataset or without a cleaned dataset, then </b>[[your journey begins at night...]] </div></div>]

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Meadow_Clipart.png?raw=true" class="journey"> <div class="introcontainer"> (align: "=><=")[## Your journey begins at dawn... <p>You are a researcher in the land of Academia. The presitigious Trials of the <font color="#C12283"><i><b>Datamancers</i></b></font> is just a few days away. You're planning on attending, and presenting some of your research. There will be hundreds of other researchers heading to these trials hoping to impress the elite Datamancers and win the Hat of Historical Analysis.</p> <p>Your research is focused on the Commandant-Lariviere family. In your research you've collected documents from the past that show the family in an area over the span of a number of years. You've recorded the year, place name, geographical coordinates, family members, and event that connects them to that place at that time (for example, a baptismal registration.) When looking over the documents, you've noticed something strange. The people who created these document noted the family's ethnicity, or race. But, about every 10 years this noted ethnicity or race changes. Taking all this information, you've created a map that documents the family's movement over time and space.</p> <p>Ready for you journey to the trails, you reach into your saddlebag one last time to look at your map, only to discover that it has been stolen!</p> #### What do you do? [[Try and track down the individual who stole your map]] [[Decide that you should build a new map.]]]</div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Journey_Midnight.png?raw=true" class="journey"> (align: "=><=")[<div class="introcontainer"> <font color="#808080"> # Your journey begins at night... <p>You are a researcher in the land of Academia. The presitigious Trials of the <font color="#C12283"><i><b>Datamancers</i></b></font>, is just a few days away. You're planning on attending, and presenting some of your research. </p> <p>You had planned to use the last day to prepare your presentation, but unfortunately you got bitten by an elephant snake, and you've been racked with fever for the last 12 hours. You still have to prepare before the Trials begin. There will be hundred of others heading to these trials hoping to impress the elite <font color="#C12283"><i><b>Datamancers</i></b></font> and win the Hat of Historical Analysis.</p> <p>You look at the moon and realize, you've only got 12 hours before you need to leave!</p> #### Have you got your data? (link: "Yes")[Great, [[head back to your hovel to pack]]] (link: "No")[Than get a move on! Head to the [[the general store]]] </font>] </div>

<div class="introcontainer"> # You open the door to your hovel... <div class="hovel"> <p>After you place your rucksack on the table, you open it and find all the data needed for the trials. Unfortunately, it is in bad shape. You collected this data by scrapping and extracting it from the magical InterWeb. Whenever this happens, often bits of HTML code, duplicate information, or extraneous information - otherwise known as netherworld residue, will cling on.</p> <p>There are lots of tools and methods that you can use to clean up your data, which will make it easier for you to analyze and manipulate in all sorts of ways.</p. <p>If you want to learn more about these kinds of tools you can checkout <a href="http://dirtdirectory.org/tadirah/cleanup" target="_blank">DiRT: Digital Research Tools</a>.</p> <p>For now, you can find some tools in your tool chest.</p> [[You open the tool chest]]. </div></div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Sack_Clipart.png?raw=true" class="rucksack" width="300px"> <div class="introcontainer"> (align: "=><=")[ ## Now that you've summoned your data, you'll want to make sure its clean and free of netherworld residue. ###It is time to start organizing and packing your [[head back to your hovel to pack]]! </div>]

(align: "=><=")[<div style="display:inline-block;"><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/door2_pixabay.png?raw=true" width="400px" align="right"><br><div class="introcontainer" style="display:inline;" align="center"><br><h3>Look at that data! It sure looks organized and clean!</h3></div></div>] <div class="introcontainer">(align: "=><=")[<p>However, there is one more step. If you really want to impress the elite <font color="#C12283"><i><b>Datamancers</i></b></font> you're going to need to make sure your locations are accurate and the best way to do this is with longitude and latitude.</p> <p>Add two more columns two your table, this is easiest to do in a spreadsheet.</p> <div class="lat"><table><tr><th> Latitude</th><th>Longitude</th><th>City</th><th>County</th><th>...</th></tr> <tr><td>51.945407</td><td>1.062086</td><td>Manningtree</td><td>Essex</td><td>...</td></tr> <tr><td>51.735587</td><td>0.468550</td><td>Chelmsford</td><td>... </td><td>...</td></tr></table></div> <p>There are a number of website that will provide coordinates for geographic locations, one option is <a href="http://itouchmap.com/latlong.html" target="_blank">iTouch Map</a>. After you've entered the longitude and latitude for each you are ready to move on to the next stage of your quest.</p> #### <p>There is enough to get some shut eye before morning, best take the time to rest now. Tomorrow, [[you will create your map|Lap-topé]]</p> ]</div>

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Old_Scroll_Clipart.png?raw=true" class="chestr" width="25%" align="right"> <h2><a href="https://www.import.io/" target="_blank">You pick up a Scroll of Scrapping!</a></h2> <p>A Scroll of Scrapping allows you to harness the power of <a href="https://www.import.io/" target="blank">Import.io</a>, which is a tool to extract data from webpages. Import.io scrapes data off webpages and turns it into a structured table of data that can be accessed through an API. An API is an Application Programming Interface that can access data in an application or service like a search engine or digital database.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/import_weme/import_io_1.png?raw=true" border="2px">] <p>The Import.io application offers four different methods of extraction: Magic, Extractor, Crawler and Connector. </p> <p>There only two Scrolls of Scrapping on the shelf, the Extractor Scroll and the Crawler Scroll, if you'd like to learn about the two missing ones you can read them <a href="http://support.import.io/knowledgebase" target="_blank">here</a>. If you choose not tp download the application and simply use Import.io through the website, your only option will be the Extractor.</p> <p>Using an Extractor Scroll, you can train Import.io to recognize data on a single webpage, extract it and format it into a table. After you train the Extractor to do this, it can be used as a Bulk Extractor, to download data from several pages with similar layouts. This works when you know the specific URLs of each webpage you want to pull data. If you don't have or know all the URLs of the pages from which you want to extract data than the Crawler Scroll will be of more use to you. </p> #### You notice a pamphlet on the shelf, the title across it reads "Visual Guide to the Extractor." (link-goto: "You pick it up.", "Extractor") <p>The Crawler Scroll will scan a page for links visit all the pages which match a particular URL pattern. Crawlers are useful when the data you are look for is spread across an entire website. It will systematically extract data from multiple webpages, unlike the Extractor, the Crawler travels across the entire website pulling out the data you need and insert it into a table.</p> #### You notice a pamphlet on the shelf, the title across it reads "Visual Guide to the Crawler." You read it. (link-goto: "You read it.", "Crawler") <p>It's an excellent tool for a novice <font color="#C12283"><i><b>Datamancers</i></b></font>.</p> #### Once you selected and purchased the Scroll of Extraction that you want to use, you can use it to summon your data. <p>Want to use a Scroll of Scrapping?</p> <p>(link: "Yes")[Great, then pick it up and [[leave the general store]]]</p> <p>(link: "No")[Oh, then lets check out the [[Scroll of Mirroring]] instead.]</p>

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Old_Scroll_Clipart.png?raw=true" class="chestr" width="25%" align="right"> <h2><a href="http://www.gnu.org/software/wget/" target="_blank">You picked up the Scroll of Mirroring!</a></h2> <p>The Scroll of Mirroring is very powerful and it calls upon the magic of Wget; if one is not careful, it can wreck havoc. Wget can download entire websites with a single command. It will copy their HTML, graphics, content, and files; creating mirror-images of the website. It is a fast way to obtain large amounts of information and allows you obtain data for your dataset using a single command.</p> <p><h4>You see a leaflet on the shelf, the title reads <a href="http://programminghistorian.org/lessons/automated-downloading-with-wget#step-one-installation" target="_blank">"Setting Up Wget."</a> You notice a sticky-note is stuck to one of the pamphlets in the pile. You read it...</h4><p>(mouseover-append: "You read it..." )[(align: "=><=")[ <h4><i>"Ignoring this leaflet, it isn't recommended. Might become a piglet... Like my friend Ben did."</i></h4>]] <p>The Scroll of Mirroring is imbued with the magic of Wget, but to properly wield it requires you to work from the command line or terminal. (Depending on whether you are Mac Magician or a Windows Wizard). You will need to understand how to contruct a command that will direct Wget to what you want to download, but you will also need to specify the depth or range which you want to Wget to travel. </p> <p>Once you have setup Wget and have opened your command line or terminal, you will want to make a directory to work from. Type in <code>mkdir (directory-name)</code>, replacing <i>(directory-name)</i>, brackets and all, with the name you want to give this directory. Proceed to this new directory by using the <code>cd (directory-name)</code>, cd is the abbreviation of change directory.</p> <p>Now that you have created a workspace, you can begin to construct your command. Your command will always begin with <code>wget</code>, as this will tell the computer that it needs to locate Wget first. This particular scroll uses a basic Wget command but there are more complex commands that can be used for more specific operations.</p> <p>The basic Wget command can be broken down in two ways: </p> <p><code>wget (operation sepecifics) (URL)</code> This first command directs Wget to the url provided and based on the operation specifications will mirror additional pages. </p> <p><code>wget (operation specifics) (file.txt)</code> This second command looks at the text file provided, in which contains a list of different URLs to create mirrors of, still with in the operation specifications.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/wget_weme/wget_multipleurls_list.png?raw=true" border="2">] <p>The specifications that can be added to a Wget command will narrow and control the actions of Wget. You can learn even more of these from the official manual that can be found <a href="http://www.gnu.org/software/wget/manual/wget.html" target="_blank">here</a>, but the following are the basic specifications that are need to download all sorts of files and webpages.</p> <p><code>--no parent</code> or <code>-np</code> tells Wget not to follow links above the specified directory level. For instance with the follow string: <code>wget -np http://www.url.com/directory_A/directory_B/</code> Wget will download all files and directories that are contained in the directory_B but it will not travel above and mirror anything from the directory_A.</p> <p><code>--recursive</code> or <code>-r</code> tells Wget to follow and mirror the links and directory structure of the website. Recursive retrival means that Wget will start by downloading the original requested page that is used in the string, it will then follow the links on the page to the next page, which downloads, and then follows that page's links to next one. It will continue to follow this path until it has reached its maximum depth (5 layers) or the specified depth with...</p> <p><code>--level=(depth)</code> or <code>-l (depth)</code> this will specify the maximum depth which the recursival retrevial descended through the website.</p> <p><code>--limit-rate=(amount)</code> this establishes a rate of download which Wget must follow. This is an important specification as it determines how much bandwidth Wget will use during the download process. <code>--limit-rate=20k</code> limits Wget to 20 kilobytes per second.</p> <p><code>--wait=(seconds)</code> or <code>-w (seconds)</code> is another important specification, it tells Wget to wait a certain numbers of seconds between retrievals. Wget accesses the websites server in order to retrieve and mirror the webpages, which can cause stress to website server. Including this specification will lighten the load and is a courtesy to other digital citizens accessing the website.</p> <p><code>--mirror</code> or <code>-m</code> is a specification that will automatically set the recursion depth to infinite, the purpose of this specification is to open up the limitations and allow for an entire website and its backend to be mirrored and downloaded. Including <code>--mirror</code> means that you don't need to include <code>--recursive</code> or set <code>--level=(depth)</code>.</p> <p>With these specifications, you can build a command that will download all of, or only a portion of a website without overloading your computer or the website's server.</p> <pre><code>wget -r -np --limit-rate=20k -w 10 http://www.url.com/directory/subdirectory/index.html</code></pre><p>This command tells W-get to follow the links and the structure of the website, it will not mirror any pages above or beyond the "subdirectory." It will only use 20 kilobytes of bandwidth per second to download the pages and will wait 10 seconds between each retrieval.</p> <p>The process of constructing a functional Wget command can be, at times, one of trial and error. Making small adjustments to the command will be necessary, especially since you want to be as specific as possible so that it doesn't get out of control.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/wget_weme/wget_brimstone/wget_attempts.png?raw=true" border="2"> <font-size="18"><i>The above image depicts the number of Wget commands attempted for this project before constructing one that worked.</i></font>] <p>There are other times when your Wget command may function perfectly, but you'll find the mirrored pages are empty or incomplete. When this happens it most likely due to the fact that the website has had a <i>Robots.txt</i> added to the backend code. This file contains The Robot Exclusion Protocol, which prevents robots like Wget from crawling their a site. To learn more about the Robot Exclusion Protcol, go to <a href="http://www.robotstxt.org/robotstxt.html">The Web Robots Pages</a>.</p> <p>If this occurs, in the directory of you Wget download you will find a file named <i>robots.txt</i>. If this is the case, you'll need to try a different command or find another method of obtain the data.</p> <p>Want to use the Scroll of Mirroring?</p> <p>(link: "Yes")[Great, then pick it up and [[leave the general store]]]</p> <p>(link: "No")[Oh, lets check out the [[Scrolls of Scrapping]] instead.]</p>

<div class="tutcontainer"><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/chest_clipartlord.png?raw=true" class="chestr" width="25%" align="left"> ## You open the tool chest... <p>Inside you find **3 different tools** that can help you clean and organize your data. Depending on the type data, as well as the state of it and where it came from, you may need all or just one of these tools.</p> ## First, lets take a closer look at each tool. ### Regular Expressions <p>A regular expression is a sequence of characters, that can be used to search text and locate patterns. With these patterns, regular expressions can also be used to manipulate data into an easier to use format. Regular expressions are compatible with certain kinds of text editors and are inputed into the Find and Replace fields.</p> ### OpenRefine <p>OpenRefine is an excellent tool for exploring large datasets in an efficient manner. It can be used on a variety of different dataset, so long as the data is already in a table or spreadsheet format. It can help to improve the accuracy and quality of that data for later analysis. It is a useful tool even for those who don't have a lot prior experience and knowledge with data cleaning.</p> ### Stanford Named Entity Recognizer (NER) <p>Stanford NER is a tool that works with Java to read through text and labels word sequences it recognizes based on the classifiers that are selected for it to look for, such as the name of a person, organization, or location. Using NER on a text will produce an output file that is a duplicate of the original text except that all of the recognized entities will have been tagged with their corresponding categorey. The ideal data for NER is unstructured data. This could be text or data that has not been organized into columns or that does not have any clearly defined pattern repetitions.</p> #### Before using any of the tools you found in the chest, you should establish what kind of data you are working with and then decide which tool you should use first. <li>Are you working with a text file or csv file? Do you need to remove unwanted punctuation, HTML coding and formatting, or any unwanted information from your data? -- If so, then you'll want to start by working with regular expressions.</li> <li>Are you working with a csv file that may contain numerous alternate spellings or lots of duplicate information? -- If so, then you'll need run it through OpenRefine.</li> <li>Maybe you have a large text file and you need to identify the name of places or dates before you can even think about removing anything with regular expressions or how many alternate spellings there might be? -- If so, then you might want to use Stanford NER first.</li> ### So, which tool do you pick up first? <p>[[Regular Expressions]]</p> <p>[[OpenRefine]]</p> <p>[[Stanford NER]]</p>

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/chest_clipartlord.png?raw=true" class="import" width="450px"> ## You picked up Regular Expressions... <p>To use regular expressions all you will need is a text editor like <a href="https://www.sublimetext.com/" target="_blank">Sublime Text</a> or <a href="https://atom.io/" target="_blank">Atom</a>, both can be used by either Windows Wizards or Mac Magicians.</p> <p>Regular expressions are strings made up of characters, metacharacters, and operators that match a pattern of tokens (e.g. word, numbers, punctuation) in a text. They can also be used to manipulate text. Regular expressions can be used in all sorts of programs, but for cleaning a text file you simply need the find/replace box in a text editor. Entering a regular expression into the find field will highlight any and all text that matches the pattern of your regular expression. After you've opened your text editor, check to see if you need to enable the "regular expressions" feature for find/replace. Since you work directly in the documents or file when you use regular expressions it is important to make a copy before you begin to make any changes.</p> <p>An excellent resource for learning and testing regular expressions is <a href="http://www.regexr.com/" target="_blank">RegExr</a>. In the image below we have copied and pasted a sample of the text into the text field, letting us test out each of our regular expressions to make sure they are finding the pattern we are looking for.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean3.png?raw=true" border="2">] <p>The tabs located at the bottom of the RegExr work area provide extra information about the regular expressions that you enter into the field above the textarea. They will be useful if you have any errors or difficulties constructing a regular expression.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean4.png?raw=true" border="2">] <p>Before testing your regular expression in RegExr, click on the <b>Flags</b> on the upper right side of the window, check both <b>global</b> and <b>multiline</b>. Activating these will make sure your pattern is tested on the entire sample text you have provided in the textarea.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean5.png?raw=true" border="2px">] <p><b>In the bottom of the chest you see a manual for this tool, the title says "Structuring the Unstructured: RegEx for NER"</b></p> { (if: (history:) contains ("RegEx_Manual1"))[You return the booklet to the chest and pick up the [[OpenRefine]] tool.] (else:)[You pick up [["RegEx for NER"|RegEx_Manual1]]]}

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/chest_clipartlord.png?raw=true" class="import" width="450px"> <h2><a href="http://nlp.stanford.edu/software/CRF-NER.shtml#Download" target="_blank">You picked up Stanford NER!</a></h2> <p>Stanford NER requries Java 8.1 or later to run. It runs through a shell such as the command line or terminal, depending on whether you are a Mac Magiciain or Windows Wizard, or it can be run through (link: "bash")[bash (find out more about bash for Macs and Windows <a href="http://programminghistorian.org/lessons/intro-to-bash" target="_blank">here</a>, Windows Wizards may want to consider download <a href="https://git-for-windows.github.io/" target="_blank">Git Bash</a>)]. </p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/NER_files/NER_runontextfile4.png?raw=true" border="2">] <p>The Named Entity Recognizer program operates by "reading" a chosen file. It identifies sequences of words that are names of locations, people, organizations. It does so by using trained classifiers and a statistical analysis of word use. The SNER package comes with 3 different classifiers:</p><div style="margin-left:50px;line-height:18.5px;"><dl> <dt><i>english.all.3class.distsim.crf.ser.gz</i></dt> <dd>Person, Location, Organization</dd> <dt><i>english.all.4class.distsim.crf.ser.gz</i></dt> <dd>Person, Location, Organization, Misc</dd> <dt><i>english.all.7class.distsim.crf.ser.gz</i></dt> <dd>Person, Location, Organization, Money, Percent, Date, Time</dd></dl></div> <p>The online demo, <a href="http://nlp.stanford.edu:8080/ner/" target="_blank">found here</a> will let you try out these different classifiers and see how NER works, before downloading it. You can use NER on a TXT file and have the result returned as a tagged TXT file or as TSV file that is ready to be opened in a spread sheet. However, whichever output is chosen, you will still need to clean the data.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/NER_files/NER_results.png?raw=true" border="2">] <p>It is important to check that you have an up-to-date version of Java on your computer, at least <a href="http://www.oracle.com/technetwork/java/javase/overview/java8-2100321.html" target="_blank">Java 8</a>. You will also need to check that its the pathway to its location has been added to your computer's environmental variables. This is how your computer will located Java when you enter the command into your shell interface, command prompt or terminal.</p> <p>Checking the version you have on your computer is very simple. The following method works for both Mac Magicians and Windows Wizards. First, open your shell interface and type <code>java -version</code>. Your computer should then respond with information about the version of Java that is installed. Java is located in a sub-directory named bin, within the main Java directory.</p> <p>Mac Magicians should <a href="http://stackoverflow.com/questions/7501678/set-environment-variables-on-mac-os-x-lion" target="_blank">click here</a> to find out how to check or add the Java path to your environmental variables, while Windows Wizards should <a href="http://www.computerhope.com/issues/ch000549.htm" target="_blank">click here</a>.</p> <p>If you think this is the tool you want to use, click the links below to learn more about it and how to install and use it: <div class="introcontainer" style="line-height:1;margin-left:25px;margin-top:-1%;margin-bottom:-2%;text-align:left;"> <li><a href="http://historyinthecity.blogspot.ca/2014/06/how-to-use-stanfords-ner-and-extract.html" target="_blank">Use NER and Extract Results</a> (for Mac Magicians)</li> <li><a href="http://imgur.com/a/yCPJx" target="_blank">Running Stanford NER on Windows</a> (for Windows Wizards)</li> <li><a href="http://www.themacroscope.org/2.0/using-the-stanford-named-entity-recognizer-to-extract-data-from-texts/" target="_blank">The Historian's Macroscope: Using the Stanford Named Entity Recognizer</a></li> <li><a href="http://ricedh.github.io/03-ner.html" target="_blank">Digital History Methods: Detecting Locations with NER</a></li></p></div> <br> #### Decided to use NER? Which kind of file did you choose as your output? <p>(link: "It was a TSV file.")[After removing the uneccessary data and columns, you'll want to use the [[OpenRefine]] tool.]</p> <p>(link: "It was a TXT file.")[You still need to structure it into a CSV file, you'll want to use [[Regular Expressions]] next.] </p> <p>(link: "I don't want to use NER")[NER not for you, eh? That's alright, take another look at [[the other tools|You open the tool chest]]]

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/chest_clipartlord.png?raw=true" class="import" width="450px"> <h2><a href="http://openrefine.org/download.html">You picked up OpenRefine!</a></h2> <p>Your data doesn't have to be perfect at this point, but it does need to be structured in a table format. Once your data has been stripped of unnecessary information and it has been organized and parsed into columns and rows, it is ready for OpenRefine.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/openrefine.png?raw=true" width="700px" border="2" alt="a power tool for working with messy data">] <p>OpenRefine is a tool that can do more than just identify duplicate records. The features it provides allow you to locate inconsistent formating, merge duplicate values and analyze the distribution of the values in your dataset. It is one-part cleaning tool and one-part analysis tool. It is a tool that makes exploring large datasets very easy.</p> <p>OpenRefine uses data transformation tools to manipulate and modify large amounts of data. This makes OpenRefine a very valuable tool for putting together a dataset that will be used for mapping. Facet, filter and cluster are features that allow you to explore and clean your data in straightforward way. They can help with finding inconsistensies in your data, whether it be spellings mistakes in placenames or alternating date formats, and helps you to find ways to resolve these inconsistencies without hindering the accuracy of your data.</p> <p>A dataset that has been constructed using a variety of different sources can be imported into OpenRefine and by using <b>cluster</b> you have control on how duplicates should be dealth with. <b>Cluster</b> lets you pick and choose which duplicate to merge and collate under a uniform valuen. It also allows you to adjust the settings to adjust how it identifies similar or duplicate values. <b>Facet</b> counts the number of unique values in a column. Using <b>facet</b>, you can discover how often a value appears in a dataset. For the purposes of mapping the travels of an individual, this feature can help to reveal movement patterns.</p> <p>If you want to learn more about exploring and cleaning data with <a href="http://thomaspadilla.org/dataprep/" target="_blank">Getting Started with OpenRefine</a>. OpenRefine also provides tutorial videos <a href="https://youtu.be/B70J_H_zAWM" target="_blank">here</a>.</p> <p>The program runs locally on your computer, but will run in your default browser. OpenRefine is a suitable tool for Windows Wizards and Mac Magicians. There are different download packages for both, as well as a download package for any Apprentices of Thaumanturge Tux. (mouseover-replace: "Apprentices of Thaumanturge Tux.")[linux users.]</p> #### How does your dataset look? Is it organized and clean? Are you ready to call it day? <p>[[Yes, it is all set for the next step!|finished packing]]</p> <p>[[I'm not sure. Let me take one more look in the tool chest|You open the tool chest]]</p> </div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/cross-296395_960_720.png?raw=true" class="skull" width="500px"> (align: "=><=")[<div class="introcontainer"> # You end up tracking down the evil villian. <p>You challenge them to a duel, and die.</p> ### Game Over #### Return to [[The Beginning]].</div> ]

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Meadow_with_Path_Clipart.png?raw=true" class="journey"> <div class="introcontainer"> (align: "=><=")[ ## You decide to build a new map... <p>With a heavy heart, you must start your research from the beginning...</p> ### But wait! <p>Of course, you remember, you have a copy of the information you used to create your map saved in your magical Lap-topé.</p> [[You run back to your hovel in search of Lap-topé]] ]</div>

<div class="introcontainer"> <div class="hovel"> # You return to your hovel... <p>After opening the door you find Lap-topé right where you left it, sitting on the bedside table.</p> <img src="https://upload.wikimedia.org/wikipedia/commons/6/66/Osborne_1_open.jpg" width="400px" class="laptop"><br><br> <p style="display:inline;">[[Turn it on.]]</p> </div></div>

<div class="introcontainer"> (align: "=><=")[<h1>After you login in...</h1> Click on <b>Maps</b> in the top right corner. <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/1.PNG?raw=true" border="2px"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/new%20map%20log.PNG?raw=true" border="2px"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/2.PNG?raw=true" border="2px"> Next, you must [[import dataset]] on "New Map." </div>]

<div class="introcontainer"> (align: "=><=")[<h1>You remember!</h1> <p> ###You used a program on the interweb called **CartoDB.**</p> <p>CartoDB is a computing platform that provides mapping and data visualization tools. Everything is hosted online, which makes it easily accessible. It is easy to use, and once you've uploaded your data it immedieately detects (and plots) geospatial data (such as longitude and latitude.)</p> <p>You type in the CartoDB link in your web browser. [[next, you must login.]]</p> <details><summary><i>Tip: Do you have a google account?</i></summary><p>If you belong to the Google club, you can use your email and password to set up an account and login!</p></details></p> ]</div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/target-335029_960_720.png?raw=true" style="float:right;z-index:-1;" width="500px"> (align: "=><=")[<div class="introcontainer"> #Feeling hopeless, you give up... <p>After giving up your career in Academia, you become immersed in the criminal underworld. Eventually you decide to become a professional hit-man.</p><p>You are never heard from again.</p> ### Game Over #### Return to [[The Beginning]].</div> ]

<div class="introcontainer"> <div class="terminal"> (align: "=><=")[<p><h1>You turn it on...</h1></p> <p>Ah-ha! You've located your information, the genealogytable. Here is what it looks like.</p> <p>See, it has the names of the family members, the documents that place them in a particular location, and the geographical coordinates for that location.</p> <p>But you still have to build your map over. You must remember how you did this...</p> [[Do you remember]] or [[do you give up]]? </font></p>]</div></div> <audio src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/iMac_Startup_Chime.wav" autoplay>

<div class="introcontainer"> (align: "=><=")[<p><h1>Import your dataset into CartoDB.</h1></p> <p>Do you want to work with the [[The Dataset of the Commandant-Lariviere Families]] or [[The Dataset of the Infamous Witchfinders]]</p> </div>]

<span style="margin-top:20px;">(align: "=><=")[<h2>The Data View</h2>]<br><br><span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span><br> <div class="tutcontainer"> <p>We can see that CartoDB has uploaded our dataset (which is currently a table.) It has also recognised the geographical coordinates of longitude and latitude in our data. This is what CartoDB will use to plot our data.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/3g.PNG?raw=true" border="2">] <p>The geographical coordinates that have been plotted on the map are associated with the column <b>the_geom</b>. When you create a map, CartoDB will ask you to assign a column as a "georeference". This is where the_geom coumn will pulls it geographical data from in order to plot the map.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_geo.png?raw=true" border="2">] ### Let's go back to [[Map View.]] </div>

(align: "=><=")[<h2>The Default Map</h2>]<br><span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span><br> <div class="tutcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/4g.PNG?raw=true" border="2">] <p>This is the map that CartoDB has automatically generated. We can see that it has plotted the geographical coordinates associated with our dataset. If you discover that your coordinates are incorrect you can easily fix this in CartoDB instead of having to fix your original file and upload all over again. Just double click on the cell in <b>the_geom</b> and fix it.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG18.png?raw=true" width="1000px" border="2">] ### Let's see what visualizations we can make with [[CartoDB|tools CartoDB]].

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Meadow_and_River_Clipart.png?raw=true" class="journey"> (align: "=><=")[<div class="introcontainer"> ### Phew, we finished this in the nick of time! You safely shutdown Lap-topé and zoom out the door! [[Head to the Trials of the Datamancer|Head to your conference.]] [[Something comes over you...|you'd rather be a pirate...]] </div>]

(align: "=><=")[<h2>CartoDB Features</h2>]<br><span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span><br> <div class="tutcontainer"> <p>CartoDB has different options to visualize data that may be better suited for dataset. Let's take a look at some of the tools CartoDB offers. We can access this by clicking on the paintbrush in our wizard tab on the right hand side.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/5g.PNG?raw=true" border="2">] ### Let's take a look at the [[Cluster]] tool. </div>

(align: "=><=")[<h2>Choose Your Map Style</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>With our basemap sqaured away, let's incorporate one of the visualization options we learned before.</p> #### Do we want to visualize our data using: (if: (history:) contains ("The Dataset of the Commandant-Lariviere Families"))[(display: "gene_data")]</p>] (if: (history:) contains ("The Dataset of the Infamous Witchfinders"))[(display: "witch_data")] </div>

(align: "=><=")[<h2>Toque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>Back at the wizard tab, we see something called Torque. That sounds fun.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/7g.PNG?raw=true" border="2">] <p>Torque gives us the option to animate our data, in this case an individual's movements across space and through time. Rather than showing us a straight line from place to place, Torque is animating our data in places to show when/where individuals are. </p> This is really nice. Do we like this, or do we want to see more? [[Yes|Heatmap]] [[No|map]] </div>

(align: "=><=")[<h2>Heatmap</h2>]<br> <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>Back at the wizard tab, we see something called "Heatmap." That sounds fun.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/8g.PNG?raw=true" border="2">] <p>In Heatmap, the more times a particular geographical coordinate appears in your dataset, the larger the marker. The marker grows in size (and deepens in colour) with the frequency of the appearance of a geographical coordinate. </p> Look at the time, it's (current-time:)! [[We should get started!|map]] </div>

(align: "=><=")[<h2>Cluster</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/6g.PNG?raw=true" border="2">] <p>Cluster is very similar to Simple, the default state which your map will upload in. Markers are plotted on your map based on the geographical reference you provide. In Cluster, CartoDB assigns a numerical value for matching data at a particular location. In some cases, this may be mulitple people at one location. This is helpful, as it gives us some understanding of how many people might be at one spot at one time. </p> Should we look at more tools? [[Yes|Torque]] [[No|map]] </div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Path_Castle_Clipart.png?raw=true" class="castle" height="1500px">(align: "=><==")[ ## You've arrived at the Trials <font color="#C12283"><i><b>Datamancers</i></b></font>, fantastic! <p>You are a little nervous, maybe you should study your work a little bit before it is your turn to go in front of the <font color="#C12283"><i><b>Datamancers</i></b></font>...</p> <p>[[Your nerves get the better of you and hide in the washroom for the rest of the day]] [[You pull out your work]]</p>]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br><br><br> <div class="introcontainer"> (align: "=><=")[ # You carefully pull out your work... <p>You are excited to debut a beautiful new map you have created, showing where these families appear over time. You created this map using ancient documents found in the elusive country archive.</p> ## You unroll The Magical Map of Commandant-Lariviere and begin to study it... <p>What kinds of things do you see when <a href="https://emilykkeyes.cartodb.com/viz/4ee8b93e-d438-11e5-af2d-0ef24382571b/embed_map" target="_blank">looking at the map</a> you've created?</p> [[It's pretty...but nothing.]] [[Over time, the family appears to be moving further West.]]] </div>

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br><br> <div class="introcontainer"> (align: "=><=")[ <p>If we look at the historical records that enumerate the family in North Bay in 1901, we see that they are living on an "Indian Reserve" outside of the city, and recorded as "Indian."</p> <img src=https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/census%201901.PNG?raw=true" border="2px"> <img src=https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/1901%20census%202.PNG?raw=true" border="2px"> [[This is interesting...]]]</div>

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ <p>This is interesting, because if we look at the other documents that enumerate them in Mattawa, they were noted as "White" and "Half Breed French."</p> <p>What is a "Half Breed"?</p> <p>From your other research, you know that this term was in the past used by census enumerators interchangably with Métis.</p> <p>In fact, looking at the 1881 Census document that the Commandant-Lariviere family appears in, you see this note:</p> <blockquote>These people are so mixed up with Indian that I scarcely know what to call them. The principle mixture is white, and they cultivate the soil, so I call them white.</blockquote> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/census%202.PNG?raw=true" border="2px"> <p>**Library and Archives Canada, RG 31, Statistics Canada, Census of Canada, 1861, C-1091, District 6, Nipissing, 42, lines 4-17.**</p> [[So what does this mean?]]]</div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Sack_Clipart.png?raw=true" class="rucksack" width="300px"> <center><div class="introcontainer"> ## You've acquired an Extraction Scroll ### Now that you are outside of the general store you can use your scroll to summon your data!(click: "summon your data!")[(display: "summon your data")]

##### As you unroll the (text-style: "blur")[scroll your vision blurs and ](text-style: "blurrier")[the world begins to fade away...] (click: "unroll the scroll")[ {(live: 0.75s)[ (transition: "dissolve")[ <img src="http://orig02.deviantart.net/e5ec/f/2007/335/8/f/ascii_kaleidoscope_animation_by_fence_post.gif" width="100%"><br> [[You successfully obtained your data.]](stop:)]]}]

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Old_Scroll_Clipart.png?raw=true" class="import" width="25%">(align: "=><=")[ ## The Crawler ] (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_6.png" border="2">] <details><summary>Step 1</summary><p style="padding-left:10px;padding-right:10px;"> The Import.io application functions just li ke a browser, you input the URL into the address bar and you are able to website functions no different than it would in browser. After opening the import.io application and selecting <b>Start Crawler</b>, you will need to navigate to the URL which you want to extract data from. Once there click on <b>I'm there</b>.</p></details> (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_7_2.png?raw=true" border="2">] <details><summary>Step 2</summary><p style="padding-left:10px;padding-right:10px;">A new button will appear, <b>Detect optimal settings</b>. This disables any javascript on a webpage to improve extraction. This could result in your data disappearing, if this happens then select <b>No</b> when asked if the data is still visible. This will reload the page with the javascript and allow you to continue. If you can still see the data you want to extractm select <b>Yes</b>.</p></details> (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_8_2.png?raw=true" border="2">]</center> <details><summary>Step 3</summary><p style="padding-left:10px;padding-right:10px;">Once the optimal settings have been chosen, you'll be prompted to decide whether you want to extract data from one row or from multiple rows. One page means only one row of data needs to be extracted, while multiple pages means multiple rows. Multiple rows were chosen for this visual guide. These rows will need to be trained. This is done by selecting the row and clicking <b>Train rows</b> in the bottom right corner. Once all the rows that you want to extract on the page have been highlighted in blue, click <b>I've got all ## rows.</b><div style="margin-left:40px;"> <details><summary><i>Tip: selecting data...</i></summary> Placing you cursor at the edges or intersections of row/column borders can help with highlighting rows if you are having difficulty. If that isn't working, try selecting all the data in a row and then click <b>Train rows</b>.</details></div></p></details> (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_12_2.png?raw=true" border="2">] <details><summary>Step 4</summary><p style="padding-left:10px;padding-right:10px;">Step 3 tells the crawler where each unique group of data is. Now that your rows have been defined you will need to add columns, click <b>+ Add columns</b>. Once you've selected a cell in the column you want to create, a dialogue box will appear in the bottom right corner. This box allows you to name the column as well as specify the kind of data that will go in that column. For example if your column contains a date or image, you can select specify the column for that data. Selecting <b>text</b> is the safest choice for most data and is more likely to preserve the original formatting of the data in that column.</p> <p>After you have determined the settings, click <b>Done</b>, this creates a column in the spreadsheet on the lower half of the window. Next select a cell on the webpage with the information you want to extract to that column and click <b>Train</b>. If everything worked correctly, the data in that column should be highlighted in bright yellow and appear in the spreadsheet on the lower half of window. Now, you'll need to train the next column, repeat the process until everything you want and then click <b>I've got what I need!</b></p> <div style="margin-left:40px;"><details><summary><i>Tip: if you make a mistake during the trains of the rows or columns...</i></summary><p style="display:inline;">If you make a mistake, there is an undo buttons in the bottom right corner that will undo your last step. However, these don't alway work as well one would like.</p><center><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/import_weme/import_io_undo.png?raw=true" border="2"></center><p>Sometimes if you make a mistake training a column clicking the undo button can cause more of problems, making it more worthwhile to delete the column by hitting the trashcan/X button and start over (Trashcan will remove all, X will delete column). Clicking on the <b>Cancel</b> button, reverts you back to previous stage. In other words, if you are at the stage or training columns and rows, hitting <b>Cancel</b> will take you back to the point where you must select whether your want to extract one row or multiple rows. </details></p></div></details> (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_11_2.png?raw=true" border="2">] <details><summary>Step 5</summary><p style="padding-left:10px;padding-right:10px;">Properly training your rows and columns can take some fiddling. You'll find that highlighting the cells or rows can be tricky. Import.io is very sensitive to location of your cursor when you to select/highlight the data you want to select the cell. A little bit of time spent playing around with import.io and testing out it's various tools will help you pick up on all its little quirks.</p> <p>Once you've trained the rows and columns to extract the data you want from the page, click on <b>"I've got all ## rows!"</b></p> <div style="margin-left:40px;"><details><summary><i>Tip: Avoid the tabs at the bottom of the spread sheet...</i></summary>These tabs are the spreadsheets containing the data from each URL you've already trained. My advice is not to touch them, fiddling with any of these spreadsheets can result in having to start over from the beginning. Things can go from bad... <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/import_weme/import_io_tip2.png?raw=true" border="2"> To worse very quickly... <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/import_weme/import_io_tip3.png?raw=true" border="2"></font></i></div></p></details> (align: "=><=")[<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/Laurel/import_weme/import_io_10_2.png?raw=true" border="2">] <p><details><summary>Step 6</summary><p style="padding-left:10px;padding-right:10px;">Before you can run your crawler, you'll need to train it to find your data on at least 5 pages. After you've finished one page, the data in your spreadsheet will disappear except for the column names and tabs at the bottom. You'll see this in the bottom right corner.</p> <div style="vertical-align:middle;padding-left:25" align="center"><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/import_weme/import_io_13_2.png?raw=true" width="320px"></div> <p>Clicking on <b>Add another page</b> will result in a dialogue box popping up which says "Go to another page..." Before you click on the <b>"I'm there button"</b> you will need to enter the next URL that contains data you want to extract. Remember! This is a Crawler, you are training it to recognize a URLs with similar templates. The next page should have a similar URL and the data should be in a similar layout. After you've trained a page or two, you'll find train the rest of the pages will be much more quickly as the Crawler learns to recognize where the data is located. </p></details></p> [[You've decided which scroll you want to use...|Scrolls of Scrapping]]

{(live: 8s)[ (text-style: "fade-in-out")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Andy-whirlpool-800px.png?raw=true" class="scroll" width="800px">]]} (align: "=><=")[ <div class="introcontainer"> # You hide in the washroom for the rest of the day... <p>Hours pass by and you haven't heard anyone enter the washroom for quite some time. You open the door of the stall you've been hiding in, that's when you hear a sloshing sound from behind you. You look down just in time to see a slimey tentacle wrap around your midsection. It pulls you back and into a whirlpool vortex that has appeared where the toilet once was.</p><p>You are never heard from again.</p> ### Game Over #### Return to [[The Beginning]]. </div> ]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br> <div class="introcontainer"> (align: "=><=")[ # You carefully pull out your work... <p>As you unroll the maps, you start to reflect back on your discovery of these infamous hunters of witches, <b> Matthew Hopkins</b> and <b>John Stearne</b>. Something had made you wonder if all the legends of their evil were true. </p> <p>Since then you've been scouring through every tome, scroll, volume, mangus opus, lexicon, codex, and vade mecum that you come across in order to learn more about them.</p> ### You unroll The Magical Maps of the Witchfinders... <p>What kinds of things do you see?</p> [[You stare blankly at the map... Nothing comes to mind|It's pretty...but nothing.]] (link: "You notice something on one of the maps")[(go-to: (either: "Hop_Map", "Stea_Map"))] </div>]

<div class="tutcontainer"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/chest_clipartlord.png?raw=true" class="import" width="450px"> ### You begin to read the manual called "Structuring the Unstructured: RegEx for NER"... <p>The following instructions will use regular expressions to structure a text file that has been tagged using NER into a list of tagged locations. Following these steps will help you learn how to clean up a unstructured text file for further use. If you would like to follow along with, speak these these magical words to <b>obtain</b> the dataset: <a href="https://github.com/rhymeswithfloral/hist57022w_project/Laurel/RegEx_AnonTrueandExact/ner_TrueandExact.txt" download>"Abrahadabra!"</a></p> <p>Using regular expressions, we will clean up this text file so that all extraneous information is deleted and only a list of locations is left. Now that you have the dataset, you can open it up in the text editor of your choice. For this guide we used Sublime Text 2.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean1.png?raw=true" border="2">] <p>Everything inserted between <code>``(...)``</code> in a regular expression is called a capture group, regular expression can be made up of multiple capture groups or have none. You can use capture groups, to group parts of the pattern in the regular expression together. This means that when you apply the quantifier in the <b>Replace</b> field, it can either be applied to the entire group or restricted to only part of the pattern.</p> <p>The locations in this text have been tagged, now they appear between <code>``<LOCATION>...</LOCATION>``</code>. This regular expression will look for every instance of <code>``<LOCATION>``</code> in the text when you enter it into the <b>Find:</b> field. Once you have typed this into the <b>Find:</b> field, your text editor will highlight the text which matches the pattern of your regular expression. </p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean6.png?raw=true" border="2px">] <p>Since the goal is to construct a list of locations, first, we need isolate each location name and this is best down moving each location name to a line of its own. To do this add a newline directly infront of our capture group with the following regular expression. (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean7.png?raw=true" border="2px">] <details><summary>Regular Expression #1</summary><pre> FIND: <code>``(<LOCATION>)``</code> REPLACE: <code>``\n\1``</code><pre></details> <p>In this regular expression, <code>``\n``</code> represents a newline and <code>``\1``</code> indicates which capturing group this newline is being placed in front of. If we had a regular expression which had multiple capture groups, each would be represented by order they appear in the string</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean8.png?raw=true" border="2px">] <p>In order to obtain a list of only locations, the extraneous information needs to be removed. Now that new lines have been created with <code>``<LOCATION>``</code> at the start, we can use an expression that will locate all lines that do not begin with <code>``<LOCATION>``</code> to locate unnecessary information. To do this the regular expression needs to look for a pattern starting at the beginning of the line, this is done by adding <code>``^``</code> to the start of the expression. By including <code>``?!``</code> within <code>``(...)``</code>, the expression will look for a pattern that <u>does not</u> match the capture group. Including <code>``.+``</code> at the of our expression tells it to capture any characters on the lines it matches.</p> <div style="padding-top:-5%;padding-left:15%;"><caption><font style="font-size:16px;"><i>roll over image to view the expression applied to the sample text and to the full text.</i></font></pre></caption> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean9.png?raw=true" class="tut1" border="2"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean10.png?raw=true" class="tut1" border="2"></div> <details><summary>Regular Expression #2</summary><pre>FIND: <code>``^(?!<LOCATION>).+``</code> REPLACE: <code>``leave blank``</code></pre></details> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean11.png?raw=true" border="2px">] <p>After removing any lines that do not begin with <code>``<LOCATION>``</code> it is time to focus on removing the information that comes after <code>``</LOCATION>``</code>. Before proceeding to this next step, remove all blank line. </p> <details><summary>Regular Expression #3</summary><pre><p> FIND: <code>``^\n``</code> REPLACE: <code>``leave blank``</code></p></pre></details> <p>In order for the regular expression to catch anything that follows <code>``</LOCATION>``</code>, what is needed first is an expression that will recognize the closing location tag. Very similar to the expression used to the open location tag, <code>``(<LOCATION>)``</code>, all that needs to be added are characters that will identify the forward slash ``/`` token. As this token is recognized by regular expression character it is a special or metacharacter which needs to be "escaped," meaning that we need to supress the associations this character has with regular expressions. This is done by placing including a backslash before the special character.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean14.png?raw=true" border="2px">] <p>To capture whatever follows <code>``</LOCATION>``</code> is simply done by adding ``.+``. The dot looks for any character: whitespace, digit, letter, or punctuation marks, and the plus symbol tells the expression to look for match 1 or more of token that preceeds it. These will capture anything found on a single line.</p> <details><summary>Regular Expression #4</summary><pre><p>FIND: <code>``(<\/LOCATION>).+``</code> REPLACE: <code>``\1``</code></p></pre></details> <p>The final result should look like this:</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/RegEx_AnonTrueandExact/RegEx_Clean16.png?raw=true" border="2px">] [[Close the booklet|Regular Expressions]]

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/pirate_boat_clker.png?raw=true" class="skull" width="700px"> (align: "=><=")[<div class="introcontainer"> ### You stop in your tracks, overcome by a desire to live a life at sea. <p>You drop everything you're carrying and sprint to the nearest harbour. It isn't long before you find a ship with a black flag, fortunately for you the captain is holding interviews for new deckhands...</p><p>After a year at sea, you die of scurvy.</p> ### Game Over #### Return to [[The Beginning]].</div> ]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ ## You look closely at the map of <a href="https://lrowe.cartodb.com/viz/919a86b8-eae5-11e5-8c7b-0e3a376473ab/embed_map" target="_blank">Matthew Hopkins</a>... <p>In the summer of 1645, we see that Hopkins travels further north than Stearne did after they split in Dedham Valley. He travels as far as Rushmere before returning to Chelmsford, for the trials of the Manningtree witches.</p> <p>Unlike John Stearne, Hopkins covered a much larger portion of East Anglia, and wasn't restricting his distance or direction he was willing to travel in. However, he also does not travel west until 1646.</p> [[You look at the other map|Stea_Map]] (if: (history:) contains ("Stea_Map"))[[[What does this tell us about the witchfinders?]]]]</div>

<div class="introcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/door2_pixabay.png?raw=true" width="500px" class="door"> ## You wake up feeling rested. <p>You get out of bed and lumber over to the pantry. Inside the pantry you find a tin can, mason jar, small pouch and a mysterious glass jug. Sleepily, you grab for one of the containers...(click-append: "containers...")[(print: "<br><p>After making yourself breakfast, you decide to sit down at the computer and begin working on the magical map you'll present at the trials. You take a sip from your cup while glancing back at the table where you left the container. The label reads: $b[$cup], you sigh with content.</p>")] #### You [[turn on your magical Lap-topé| turn on 2]]. (set: $cup to (either: "coffee", "tea", "cocoa", "slime", "the blood of my enemies", "cottage cheese")) (set: $b to (text-style: "bold"))]

<div class="introcontainer"> (align: "=><=")[<p><h1>You turn it on...</h1></p> <p>Now you have to build your map. Do you remember the name of the program you were going to use?</p> <p>[[You remember!|Do you remember]] [[Nope, you give up.|do you give up]]</p> ] <audio src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/iMac_Startup_Chime.wav" autoplay>

(align: "=><=")[<h2>The Basemap.</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ### Let's try changing the Basemap. <p>Similar to CartoDB's visualization options, CartoDB also offers the option of changing the basemap. A basemap is the map on which all your data will be plotted. Depending on what kind of data you are plotting, you might want a different basemap. For example, if you are interested in plotting the movement of whale populations, you probably want a basemap that highlights oceans, over a basemap that higlights roadways. Let's start making our map by choosing an appropriate basemap. Let's click on <b>Change basemap</b> in the bottom left corner.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/9.PNG?raw=true">] <p>See, there are different options we can choose for our map, including night-time maps, watercolour maps, and we even have the option of uploading our own basemap.</p> (if: (history:) contains ("The Dataset of the Commandant-Lariviere Families"))[ ### Do we like [[Watercolour|Edit the Bubbles]] (align: "=><==")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/watercol.PNG?raw=true">] ### Or [[Dark Matter|dark matter]]? (align: "=><==")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/dark.PNG?raw=true">]] (else-if: (history:) contains ("The Dataset of the Infamous Witchfinders"))[<p>The Witchfinders dataset has several references to towns and places that no longer exist, or have since been amalgamated into larger towns and cities. Let's start plotting the map before making a final decision about what background to use.</p> Let's move on to the next step and [[Edit the Bubbles]].] </div>

(align: "=><=")[<h2>Dark Matter</h2>]<br> <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Hmm... <p>The one thing about the dark matter basemap is that it is a bit dark. It's a bit difficult to see precise locations. This kind of map would be beneficial to see data like energy uses, but to see the locations of our families, it isn't as useful.</p> Let's go with a watercolour basemap [[instead|Edit the Bubbles]]. <div>

(align: "=><=")[<h2>Torque</h2>]<br> <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>In the wizard for Torque, we want to make sure we have our "year" column selected under "Time Column." This will let us move forward through time.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/12.g.PNG?raw=true">] <p>As our shape marker, we can have an [[ellipse]] or a [[rectangle]].</p>

(align: "=><=")[<h2>Cluster</h2>]<br> <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>What colour should our bubble be, [[orange]] or [[green]]?</p> </div>

(align: "=><=")[<h2>Heatmap</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/Heatmap.21.PNG?raw=true">] #### <p>Should it be animated, like Torque?</p> [[Yes make me animated|animated 21]] [[No, let's go|go]] </div>

(align: "=><=")[<h2>Heatmap</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/Heatmap.33.PNG?raw=true">] #### <p>Should it be animated, like Torque? </p> [[Yes make me animated|animated 33]] [[No, let's go|go]] </div>

(align: "=><=")[<h2>Heatmap</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/animated.21.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<h2>Heatmap</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/animated.33.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<h2>Cluster</h2>] <span style="padding-left:130px;font-size:30px;" <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/green.PNG?raw=true">] #### [[Now, let's go|go]].

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/blue.ellipses.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/red.ellipses.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/blue.rectangle.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Emily/adds/red.rectangle.PNG?raw=true">] <h3>[[Now, let's go|go]]</h3> </div>

(align: "=><=")[<div class="introcontainer"> (set: $ogre to (transition: "shudder")) (set: $water to (transition: "shudder")) ##Nothing... <p>After making this discovery, you begin to pack up. You feel the table vibrate. You look down at your glass of water and you notice it {(live: 2s)[$water[shudders.]]} {[ ]<timed-append| (set: $delay to 2s) (live: $delay * 1)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 2)[(stop:)(append: ?timed-append)[Are those footsteps?]] (live: $delay * 3)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 4)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 5)[(stop:)(append: ?timed-append)[<br>You look up]] (live: $delay * 6)[(stop:)(append: ?timed-append)[(text-style: "bold")[, a hungry Ogre looms before you! $ogre[<img src="http://img04.deviantart.net/dd57/i/2015/182/1/8/obvious_troll____or_ogre__by_pkoller-d8zi401.jpg?raw=true" width="1000px">]]]] (live: $delay * 7)[(stop:)(append: ?timed-append)[<h4>He eats you.</p> ### Game Over #### Return to [[The Beginning]]. ]]}</div>]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ <h3>Great! What else?</h3> <p>Mattawa appears to be a significant location for this family.</p> ###While maps like the one we've created are very useful... <p>They need to be analyzed in conjuncture with our data.</p> <p>If we look closer at our data, many of the individuals celebrate significant life events in Mattawa, including baptisms, and marriages.</p> <p>However by the early 1900s, the family has moved away from the area, closer to North Bay, and are living on the Indian Reserve.</p> [[Great! What else do you see?|Great! What else?]]] </div>

(align: "=><=")[<h2>Simple</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>The Simple map style is great for comparing the movements of John Stearne and Matthew Hopkins. With this style the focus is more on the locations that they travelled to, while the animated map, Torque, is ideal for fouscing the journey rather than the destination.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_screencap1.png?raw=true" border="2">] <p>CartoDB provides the option of layering multiple datasets over on another on one map. This is excellent for doing comparisons. First, we'll need to use to different markers colours to distinguish Hopkins's movements from Stearne's.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_mh3.png?raw=true" border="2">] <p>Purple and green is a good colour combo and they stand out nicely on this map background. Purple for Hopkins and [[Green for Stearne.|greent]].</p> </div>

<div class="introcontainer"> (align: "=><=")[<p><h2>You've picked the dataset you want to import.</h2></p> <p><code><font color="#FFFFFF"> {[ ]<timed-append| (set: $delay to 90ms) (live: $delay * 1)[(stop:)(append: ?timed-append)[I]] (live: $delay * 2)[(stop:)(append: ?timed-append)[m]] (live: $delay * 3)[(stop:)(append: ?timed-append)[p]] (live: $delay * 4)[(stop:)(append: ?timed-append)[o]] (live: $delay * 5)[(stop:)(append: ?timed-append)[r]] (live: $delay * 6)[(stop:)(append: ?timed-append)[t]] (live: $delay * 7)[(stop:)(append: ?timed-append)[`` ``]] (live: $delay * 8)[(stop:)(append: ?timed-append)[r]] (live: $delay * 9)[(stop:)(append: ?timed-append)[e]] (live: $delay * 10)[(stop:)(append: ?timed-append)[q]] (live: $delay * 11)[(stop:)(append: ?timed-append)[u]] (live: $delay * 12)[(stop:)(append: ?timed-append)[e]] (live: $delay * 13)[(stop:)(append: ?timed-append)[s]] (live: $delay * 14)[(stop:)(append: ?timed-append)[t]] (live: $delay * 15)[(stop:)(append: ?timed-append)[`` ``]] (live: $delay * 16)[(stop:)(append: ?timed-append)[i]] (live: $delay * 17)[(stop:)(append: ?timed-append)[n]] (live: $delay * 18)[(stop:)(append: ?timed-append)[i]] (live: $delay * 19)[(stop:)(append: ?timed-append)[t]] (live: $delay * 20)[(stop:)(append: ?timed-append)[i]] (live: $delay * 21)[(stop:)(append: ?timed-append)[a]] (live: $delay * 22)[(stop:)(append: ?timed-append)[t]] (live: $delay * 23)[(stop:)(append: ?timed-append)[i]] (live: $delay * 24)[(stop:)(append: ?timed-append)[n]] (live: $delay * 25)[(stop:)(append: ?timed-append)[g]] (live: $delay * 26)[(stop:)(append: ?timed-append)[.]] (live: $delay * 27)[(stop:)(append: ?timed-append)[.]] (live: $delay * 28)[(stop:)(append: ?timed-append)[.]] (live: $delay * 29)[(stop:)(append: ?timed-append)[.]] (live: $delay * 30)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 31)[(stop:)(append: ?timed-append)[.]] (live: $delay * 32)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 33)[(stop:)(append: ?timed-append)[.]] (live: $delay * 34)[(stop:)(append: ?timed-append)[ ]] (live: $delay * 35)[(stop:)(append: ?timed-append)[.]] (live: $delay * 36)[(stop:)(append: ?timed-append)[.]] (live: $delay * 37)[(stop:)(append: ?timed-append)[.]] (live: $delay * 38)[(stop:)(append: ?timed-append)[`` ``]] (live: $delay * 39)[(stop:)(append: ?timed-append)[<br>]] (live: $delay * 40)[(stop:)(append: ?timed-append)[[[Dataset connected|dataset connected]]]]} <br> </font> </code></p> </div>]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ ## You look closely at the map of <a href="https://lrowe.cartodb.com/viz/288b4fd8-ebd8-11e5-a983-0e3ff518bd15/embed_map" target="_blank">John Stearne</a>... <p>John Stearne's path kept him near his home, Lawshall, after he and Matthew Hopkins split-up at Dedham Valley in 1645. He doesn't travel west until 1646. This may have been because he had just become a father or because the rumours of witches led him in another direction.</p> <p>However, it could also be because of the English Civil War that was taking place at the time. In 1645, a major battle took place at Naseby in Northamptonshire which approximately 30 km from Thrapston, a town he would visit in 1646.</p> [[You look at the other map|Hop_Map]] (if: (history:) contains ("Hop_Map"))[[[What does this tell us about the witchfinders?]]]</p>]

<span style="margin-top:20px;">(align: "=><=")[<h2>The Data View</h2>]<br><br><span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span><br> <div class="tutcontainer"> <p>We can see that CartoDB has uploaded our dataset (which is currently a table.) It has also recognised the geographical coordinates of longitude and latitude in our data. This is what CartoDB will use to plot our data.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_mh.png?raw=true" border="2">] <p>The geographical coordinates that have been plotted on the map are associated with the column <b>the_geom</b>. When you create a map, CartoDB will ask you to assign a column as a "georeference". This is where the_geom coumn will pulls it geographical data from in order to plot the map.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_geo.png?raw=true" border="2">] ### Let's go back to [[Map View.]] </div>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_purple.png?raw=true" border="2"> ### Next, let's [[add layers to the map]]. </div>

(align: "=><=")[<h2>Adding Layers</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh2.png?raw=true" border="2">] <b>CartoDB lets you layer multiple datasets over one another.</b> <p>To add a new layer to your map, first you select <b>Add layer</b> tab, on the right hand side, in toolbox window. If you already uploaded another dataset you want to add, you can select it from the list that will pop up, or you can upload a new dataset to CartoDB by clicking on <b>Connect dataset</b>.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh4.png?raw=true" border="2">] <b>CartoDB also offers you the oppurtunity to add other kinds of layers to your map.</b> <p>Unconnected dots on a map can be confusing to follow, especially when there are a lot of markers on a map, and hard to analyze. Another feature of CartoDB is the ability to add new features to the map, such as lines or polygons. Creating a path using a lines layer will help to visualize the movements because it allows to establish a sequence or order to their movements. In order to build a path, you need to start an empty layer. After clicking the <b>Add layer tab</b>, instead of selecting any of the dataset in your list.</p> <p>Click on <b>Add an empty layer</b>, on the right-hand side. </p> <p>It is important to know that CartoDB has a limit on the number of layers you can add to a map for free. However, it does have an option that allows you to merge your layers together. This can be tricky to make work without losing important information and you have to be careful about which layers you choose to merge.</p> #### Now, [[add an empty layer|add]]. </div>

(align: "=><=")[<h2>Adding Layers</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> (align: "=><=")[<h3>Now you have added an empty layer to your map!</h3>] (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh5.png?raw=true" border="2">] <p>To start adding lines to it, click on the <b>Add feature</b> button, this can be found at the bottom the toolbox for the active layer.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh6.png?raw=true" border="2">] <p>Next you will be prompted to select which feature you want to add to this layer. A layer can only have one kind of feature on it, it is not possible to add geographic points and lines to the same layer. Since the goal is to visualize the route which Stearne and Hopkins took, select lines.</p> #### Next, [[add lines|lines]]. </div>

(align: "=><=")[<h2>Adding Lines</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> ### You are ready to draw the path! <p>The first thing that you will need to do is to mark the starting point of your path on the map. Try to place your lines as close to the marker of the coordinates as you can. The accuracy of this placed is improved as the zoom is increased.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh7.png?raw=true" border="2">] <p>You can draw your line one connection at a time, or you can make several connections in one line. The upside to connecting multiple locations in one line is that it is faster. The downside is that if you accidentally click "discard" rather than "done" you'll have to redo the connection for multiple towns. I would suggest making three connections at a time.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh8.png?raw=true" border="2">] <p>Having trouble finding the next marker? Using search, we can locate points for line placements.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh11.png?raw=true" width="650" border="2">] <details><summary><i>Tip: Keeping track of your route</i></summary>It is good to keep a copy of your spreadsheet open, so that you can easily check the sequence of locations visited as you draw the route.</details> #### Let's [[edit the lines]]. </div>

(align: "=><=")[<h2>Editing Lines</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span><br> <div class="tutcontainer"> <p>Changing the colour of the lines is just like changing the colour of the markers, to do so you use the toolbox on the right. Keep things simple by matching the colour of the marker to colour of the path. You can edit the opacity of your markers and your lines as well, to make them or less transparent.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh9.png?raw=true" border="2" >] <p>When zoomed out from the map the lines of the path may appear connected...</p></div><div style="padding-top:-5%;padding-left:27%;"><caption><font style="font-size:16px;"><i>roll over image to enlarge.</i></font></caption> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh12.png?raw=true" width="500px" border="2"> <img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh13.png?raw=true" class="cart1" border="2" ></div><div class="tutcontainer"> <p>But when you zoom in, you may find that they actually aren't. You can fix this by switching to lines layer and double-clicking on the line you want to edit.</p> <p>Instead of moving the line, you can edit the placement of your points, so that they aren't overlapping each other, or overlapping the marker of the over dataset. It also is useful to move you markers if you find that lines on one layer are over lapping the path of the other lines layer.</p> <p>You can also edit the location markers this way.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/d046e30574b04e5a9fb530c7de1208d6fa15404b/Laurel/cartodb_mh10.png?raw=true" border="2">] <p>If you'd rather move your geo-coordinate marker instead of your line, switch to the layer the contains the marker you want to edit and double click the marker. (if: (history:) contains ("greent"))[ #### [[Let's see how the map looks so far...|Stea_simple]] ] (else-if: (history:) contains ("purple"))[ ####[[Let's see how the map looks so far...|Hop1_Done]] ]

(align: "=><=")[<h2>Simple</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>After you repeat the process of adding a line layer for both Stearne's route and Hopkin's route your map will look like this...</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_screencap2.png?raw=true" border="2">] <p>This looks great! Why don't we try changing the background?</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG22.png?raw=true" border="2">] <p>Let's use the example discussed earlier, watercolour!</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG30.png?raw=true" border="2">] <p>The place names of the towns and counties are gone, but fortunately with the Simple map style place names can be added so long as you've include a column in your dataset with the name for each of the geographic locations.<center><table style="top:0px;bottom:0px;"><tr><th><div style="margin:0px;display:inline;float:left;"><font-size="5px">▲</font></div></th><th></th></tr><tr><td><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG24.png?raw=true" height="600" border="2"></td><td><img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG26.png?raw=true" height="600x" border="2"></td><tr><td></td><td><div style="margin:0px;display:inline;float:right;"><font size="5px"><b>▼</b></font></div></tr> </table></center> Select the column that you want to use for your labels. If you have a location that appears multiple times in your dataset the name will appear multiples times on the map, overlapping each other. To get rid of overlapping labels set the <b>Label Overlap</b> to false.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG29.png?raw=true" border="2">] <p>Additional customization that CartoDB offers include adding elements to your map like a fixed title or location annotations. Most of these changes are useful if you plan to make the map available to an audience and your want to limit or enhance their interaction with the map: add a map legend, add/remove zoom tools, add/remove a layer selector or add/remove a share tool.</p> <p>Now that the magical map is finish, lets [[go to the the Trials of the <font color="#C12283"><i><b>Datamancers</i></b></font>|go]]</p> </div>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/rosmarle-voegtll_stageg.jpg?raw=true" class="stage"> <div class="introcontainer"> (align: "=><=")[ <font color="#FFFFFF"> ## The sound of the gong signals your turn. <p>You walk through the curtains and are quickly blinded by a spotlight. If you squint, you are able to make out the <font color="#C12283"><i><b>Datamancers</i></b></font> sitting in the rows in front of the stage. This isn't exactly the setup you were expecting, oh well. You take a deep breath and [[begin to speak...]]</p> </font>]</div> <audio src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/info%40in-tune-nl_gong.mp3" autoplay>

<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/rosmarle-voegtll_stageg.jpg?raw=true" class="stage"> <div class="introcontainer"> <font color="#FFFFFF"> (align: "==><==")[ # Congratulations! <span><img style="vertical-align:middle" src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/ImgSources/Magic_Hat.png?raw=true" width="150px" class="hat"><p>The <font color="#C12283"><i><b>Datamancers</i></b></font> are immpressed by both your map, and your analysis! They have bestowed you with the title of <font color="#C12283"><i><b>Datamancer</i></b></font> and you are awarded the Hat of Historical Analysis. The <font color="#C12283"><i><b>Datamancers</i></b></font> finish their applause and come join you on stage to personally congragulate you. Many of them also talked about different mapping tools available online, including: <div class="introcontainer" style="font-size:32px;margin-top:-3%;margin-bottom:-2%;text-align:left;">(align: "=><=")[ <li><a href="http://palladio.designhumanities.org/" target="_blank">Palladio</a></li><li><a href="https://support.google.com/mymaps/answer/3024454?hl=en"target="_blank">Google Maps</a></li><li><a href="http://leafletjs.com/"target="_blank">Leaflet</a></li>]</div></p> <p><h4>Congratulations on a job well done. Time now to return [[home.|The Beginning]]</h4></p>] [[Or is it...]] </div>

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ ## After your examination of the two maps... <p>Based on the examination of those maps we have learned that between 1645 and 1648, they did most of their travelling separately. They travelled together from Manningtree to Dedham Valley, from Ipswich to Bury St.Edmunds, and from Over to Kimbolton. Compared with the distance and amount of time that was spent travelling separately, they travelled together less than half the time.</p> <p>Now using the layers of the Simple map, you can examine their path in comparison to a sample of witches that they encountered during their travels.</p> <iframe width="100%" height="520" frameborder="0" src="https://lrowe.cartodb.com/viz/8c6e26b4-efa3-11e5-acac-0e3ff518bd15/embed_map" allowfullscreen webkitallowfullscreen mozallowfullscreen oallowfullscreen msallowfullscreen></iframe> <caption><font style="font-size:16px;"><i>This is a functional map, try changing the layers with the tool in the top right corner.</i></font></caption> #### What else do we learn? <p>On this third map, there are places which they probably visited but the data acquired to build the maps do not account for them, such as March and Hitcham.</p> #### [[What do you think about mapping data?]]</div> ]

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> <p>CartoDB does not allow more than one animated layer on a Torque map. Choosing to use Torque means that you'll need to create two different maps if you want to animate the movements of both the Witchfinders, Matthew Hopkins and John Stearne.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_screencap1.png?raw=true" border="2">] <p>However, it is still possible to include more than one dataset to show the movements of each person on one map. This can be done by using multiple layers. But first, you need to use to different markers colours to distinguish Hopkins's movements from Stearne's.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_mh3.png?raw=true" border="2">] <p>Purple and green are a good colour combo and they stand out nicely on this map background. [[Purple for Hopkins|purple]] and Green for Stearne.</p>

(align: "=><=")[<h2>Simple</h2>] <span style="padding-left:130px;font-size:30px;"><code>https://cartodb.com/</code></span></span> <div class="introcontainer"> ##Here it is! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_green.png?raw=true" border="2">] ### Next, let's [[add layers to the map]]. </div>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> ### One route finished, one more to go! (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG17.png?raw=true" border="2">] <p>Now that you have one path done, repeat the same process for the other path.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG25.png?raw=true" border="2">] <p>Since the routes of both Stearne and Hopkins are on this map, to improve clarity, edit the opacity of the non-animated route so that it does not detract from the animated torque layer. (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG19.png?raw=true" height="600px" border="2">]</p> <p>Once both routes are done it is time to setup the Torque layer. As discussed earlier, Torque animates the sequence of markers and with the addition of the lines layer this animation will be easier to follow. You can also change the speed of the animation as well add a trailing markers. All this can be done in the toolbox on the right side. (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG28.png?raw=true" border="2">] The <b>Steps</b> option, for Torque, is useful when you have more a specific and consistent set of date/time data. Adjusting the steps will effect the transition of the animation from one marker to the next. When you have large amounts of data, the ability to edit the <b>Duration</b> of the animation can be useful if you want to speed up or slow down the transition. Now, what happens if we [[change the base map]]...</p>

(align: "=><=")[<h2>Torque</h2>] <span style="padding-left:130px;font-size:30px;"><code color="#000000">https://cartodb.com/</code></span></span> <div class="introcontainer"> ## Changing the basemap... <p>The map looks great! Why don't we try changing the background?</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG22.png?raw=true" border="2">] <p>Oh dear. All the place names are gone... Are we able to add them to the map?</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG30.png?raw=true" border="2">] <p>It appears that any of the visualization styles which involve animation don't have a label feature for markers. If you are willing to give up animation for labels and the watercolour background than you may want to switch to the [[Simple Map|Stea_simple]]. Let's switch back to the default basemap, Nokia Reduced Day.</p> (align: "=><=")[<img src="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Laurel/cartodb_MG31.png?raw=true" border="2">] <p>That's better. The maps look great! I think it is time to [[go]] to the Trials of the <font color="#C12283"><i><b>Datamancers</i></b></font>!

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br><br> <div class="introcontainer"> (align: "=><=")[ ###What can learn about witchfinders from this map? <p>Matthew Hopkins lived in Manningtree, however, the map shows that he did not return to Manningtree much between 1645 and 1648. This could suggest that Hopkins may have had few familial obligations or responsibilities, or that he felt little connection to his home in Manningtree, unlike Stearne.</p> <p>Stearne, who travels to and from his home is Lawshall multiple times. He owned land in Lawshall and was a new father; his daughter, Anne, was born at the beginning of 1644.</p> [[What does this mean?]]</div>]

<img src="https://raw.githubusercontent.com/rhymeswithfloral/hist5702w_project/master/ImgSources/Old_Papyrus_Clipart.png" align="left" class="scroll"><br> <div class="introcontainer"> (align: "=><=")[ ##So what does this mean? <p>It's likely then that the family is what we would consider now as Métis. This is also supported by some of your other secondary research, which talks about how government officials were often frustrated when it came to recording the ethnicity of the Métis population.</p> ### So what does the map tell us about this family then? <p>Well, it seems to suggest that despite dispering over time, Mattawa remains an important center for this family over generations.</p> <p>Additionally, their recording in documents is typical of other Métis populations, being neither one or the other. And, perhaps Mattawa is a significant community for Métis populations in the area?</p> ### What else have you learned about data visualizations? [[They're awesome. They make your presentation look so cool.]] [[There are many options available.]]] </div>

{ (if: (history:)'s length > 0)[(link: "↶ back")[(goto: (history:)'s last)]]</h4> } { (print: "<script>$('html').removeClass(\)</script>") (if: (passage:)'s tags's length > 0)[ (print: "<script>$('html').addClass('" + (passage:)'s tags.join(' ') + "'\)</script>") ] }

<br>(background: "#FFF8F2")[<div style="display:inline-block;"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="left" class="pillar"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="right" class="pillar"> (align: "=><=")[<h2>The Trials...</h2>]<div class="introcontainer" style="display:inline;"> (align: "=><=")[<p>You've been waiting in the great hall for awhile now, you've seen many other hopefuls come and go. Some carrying proudly showing off their Hats of Historical Analysis, and others fleeing the castle with tears in their eyes.</p> <p>It will be your turn in front of the <font color="#C12283"><i><b>Datamancers</i></b></font>. For your trial, you'll talk about your research, how you built your map, and then analyze your findings.</p> <b>Which map do you want to present?</b> [[The Magical Map of Commandant-Lariviere Families]] [[The Map of the Infamous Witchfinders]] ]</div> </div>]

<br>(background: "#FFF8F2")[<div style="display:inline-block;"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="left" class="pillar"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="right" class="pillar"> <div class="introcontainer" style="display:inline;">(align: "=><=")[ ### It is important to remember is that no data visualization is simply created. Each is created with a specific purpose in mind. <p>In our cases, we wanted to track the movement of people over an area and time. We ended chosing CartoDB for the tool, but we could have done lots of different things with our data, depending on what kind of end-product.</p> <p>We could have used lots of different visualization tools on our data, including: <a href="http://programminghistorian.org/lessons/creating-network-diagrams-from-historical-sources" target="_blank">Network NDagrams</a>.</p> <p>Remember, even a chart is a type of data visualization.</p> You are feeling prepared for the trials. Are you ready to [[step through the curtains]]?]</div>]

<br>(background: "#FFF8F2")[<div style="display:inline-block;"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="left" class="pillar"><img src="https://pixabay.com/static/uploads/photo/2014/04/02/10/15/column-303263_960_720.png" height="800px" align="right" class="pillar"> (align: "=><=")[<h2>What do you think about mapping data?</h2>]<div class="introcontainer" style="display:inline;"> (align: "=><=")[ <p>There are many different ways to map and each method can lead to a different interpretation of you dataset.</p> <p>They are great for presenting data, but you might be wondering if they are really all that useful for research. That greatly depends on the data you are working with. In this case, being able to do produce a map of our data and cross-reference it with other source helped with finding holes in our dataset.</p> <p>And don't forget to [[consider]]...</p>]]

(align: "=><=")[<div class="introcontainer"> # You've arrived at the General Store <p> In the general store you will get the tools you need to get your data from the magical InterWeb.</p> <p>The owner of the store greets you with a smile and nod before returning to the scroll he was reading. It is a small store and it has a long wooden shelf, dividing the store into two aisles</p> <p>There is a sign hanging on the shelf. It reads: "Scrolls of Scrapping" with an arrow pointing to the left and below that, it reads: "Scrolls of Mirroring," with another arrow points to the right.</p> <img src="https://pixabay.com/static/uploads/photo/2016/03/09/16/28/frame-1246811_960_720.jpg" width="250px"> ### Scrapping or Mirroring <p>Only two methods of obtaining data are offered here, but there are many different ways of mining the internet for data. How you go about obtaining your data is typically determined by the kind of data you want and the source of it. Both of the methods that will be discussed do require the downloading of software and set-up before they can be used.</p> <p>Interested in mining data from a searchable database? Try the [[Scrolls of Scrapping]]!</p> <p>Want to download data from several different kinds of webpages, and sources? Something more flexible and customizable like the [[Scroll of Mirroring]] might be for you!</p> <p>All your data has been made available to download? What are you doing here, [[go back to your hovel|head back to your hovel to pack]].</p> </div> ]

[[Simple|Cluster2]] or [[Torque|Torque2]]?

[[Torque|Torque1]], [[Heatmap|Heatmap1]], or [[Cluster|Cluster1]]?

<div class="tutcontainer"><p>You stuff your work back into your bag and walk off stage and through the side door. Heading back towards the great hall, your face beams with pride as you look at the Hat of Historical Analysis, proof that you have earned the title <font color="#C12283"><i><b>Datamancer</i></b></font>.</p> <p>A glimmer catches your attention as you approach the archway leading into the great hall. You turn and notice that the tapestry on the wall is covering something metallic. If it weren't for the direction you were walking it is unlikely you would have noticed it at all. (link-reveal: "You pull back the covering")[ and reveal a wall-mounted vault.</p> <img src="http://www.clker.com/cliparts/3/a/5/9/14246848101674100523vault_door2-hi.png" align="right"> <p>Opening it requires a 4 digit code.</p> <p>You look around, there is no one else in the corridor. Do you want to try opening it?<p> (link: "Yes")[(display: "code")] (link: "No")[(print: "You turn back, walk through the archway, into the hall and head [[home|The Beginning]].")] ]</div>

<div class="tutcontainer"> {(align: "=><=")[ Vault Combination(mouseover: "Vault Combination")[<div class="combo">Enter 1 digit at a time. <i>Anyone in class HIST5702w knows the answer.</i></div>]] } (align: "=><=")[<input type="number" onchange="c(this)" /><span id="link" style="display: none;"><input type="number" onchange="o(this)" /><span id="link2" style="display: none;"><input type="number" onchange="d(this)" /><span id="link3" style="display: none;"><input type="number" onchange="e(this)" /> <span id="link4" style="display: none;"> You hear a click, the lock has released and now the vault door can be opened. [[Open it.]] ] <script type="text/javascript"> function c(element){ var wort = element.value; if (wort.toLowerCase() == "5") { // Display the hidden link. $('#link').show(); } } function o(element){ var wort = element.value; if (wort.toLowerCase() == "7") { // Display the hidden link. $('#link2').show(); } } function d(element){ var wort = element.value; if (wort.toLowerCase() == "0") { // Display the hidden link. $('#link3').show(); } } function e(element){ var wort = element.value; if (wort.toLowerCase() == "2") { // Display the hidden link. $('#link4').show(); } } </script>

<div class="introcontainer"> ## Bibliography ### Archival & Data Sources <li>Ancestry.com. <i>Ontario, Canada, Catholic Church Records (Drouin Collection), 1802-1967</i>[database on-line]. Gabriel Drouin, comp. Drouin Collection. Missions de St Esprit, St Léonard, St Gabriel, 1900-1944. Page 49, entry 479, Felix Lariviere. </li> <li>Ancestry.com. <i>Ontario, Canada, Catholic Church Records (Drouin Collection), 1802-1967</i>[database on-line]. Paroisse de Immaculee Conception, 1889-1909. Page 39, entry Felix Lariviere and Jane Rastoul.</li> <li>Ancestry.com. <i>Ontario, Canada, Catholic Church Records (Drouin Collection), 1802-1967</i>[database on-line]. Mattawa, Ont, 1860-1875. Page 35-36, entry Philomine Elisabeth Lariviere.</li> <li>Ancestry.com. <i>Ontario, Canada, Catholic Church Records (Drouin Collection), 1802-1967</i>[database on-line]. Mattawa, Ont, 1860-1875. Page 18-19, entry Pierre Alexandre Lariviere. </li> <li>Ancestry.com. Archives of Ontario. <i>Registrations of Deaths, 1869-1938.</i> MS 935, reels 1-615. District of Nipissing, Division of North Bay. Page 151, entry Simon Lariviere.</li> <li>Ancestry.com. <i>Canada, Soldiers of the First World War, 1914-1918</i> [database on-line]. Library and Archives Canada. Record Group 150, Accession 1992-93/166, Box 4930 - 35. Simon Lariviere, Regiment Number 3040185.</li> <li>Ancestry.com. <i>Ontario, Canada, Catholic Church Records (Drouin Collection), 1802-1967</i> [database on-line]. Gabriel Drouin, comp. Drouin Collection. Mattawa, Ont, 1860-1875. Page 12-13, entry Joseph Lariviere and Sabet Skwewic.</li> <li>Ancestry.com. <i> Ontario, Canada, Marriages, 1801-1928</i> [database on-line].</i> Archives of Ontario, Series MS932, Reel 45. District of Nipissing. Page 45, entry Elizabeth Lariviere and Frances Doucette. </li> <li>Ancestry.com. <i> Ontario, Canada, Marriages, 1801-1928</i> [database on-line].</i> Archives of Ontario, Series MS932, Reel 66. District of Nipissing. Page 14, entry Catherine Lariviere and Angus McLeod.</li> <li>Ancestry.com. <i> 1881 Census of Canada</i> [database on-line]. Library and Archives Canada. Statistics Canada Fonds, Record Group 31-C-1. Unorganized Territory, Renfrew North, Ontario, C_13235, Page 18, lines 15-25. </li> <li>Ancestry.com. <i> 1901 Census of Canada</i> database on-line]. Library and Archives Canada. Statistics Canada Fonds, Record Group 31-C-1. Beaucage (Indian reserve/Réserve indienne), Nipissing, Ontario, Page 1, lines 36-40. </li> <li>Ancestry.com. <i> 1901 Census of Canada</i> database on-line]. Library and Archives Canada. Statistics Canada Fonds, Record Group 31-C-1. Beaucage (Indian reserve/Réserve indienne), Nipissing, Ontario, Page 4, lines 44-50.</li> <li>Ancestry.com. <i> 1881 Census of Canada</i> [database on-line]. Library and Archives Canada. Statistics Canada Fonds, Record Group 31-C-1. Unorganized Territory, Renfrew North, Ontario, C_13235, Page 5, lines 1-12. </li> <li>Author Unknown. "A True and Exact Relation of the severall Informations, Examinations, and Confessions of the late Witches, arraigned and executed in the County of Essex" London: 1645. From Division of Rare & Manuscripts Collections, Cornell University. ``<http://ebooks.library.cornell.edu/cgi/t/text/text-idx?c=witch;idno=wit094>``</li> <li>Carlson, Marc. "Witches and Witchtrials in England, the Channel Islands, Ireland and Scotland" <i>I. Marc Carlson.</i> Last updated July 12, 2004. ``<http://www.personal.utulsa.edu/~marc-carlson/witchtrial/eis.html>``</li> <li>Gaskill, Malcolm. <i>The Witchfinder: a seventeenth-century English Tragedy.</i> London: John Murray, 2005.</li> <li>Hamilton, Michelle. "'Anyone not on the list might as well be dead": Aboriginal Peoples and the Censuses of Canada, 1851-1916." <i>Journal of theCanadian Historical Association 18</i> no. 1 (2007): 57-79.</li> <li>Hogue, Michel. "Exile: Scrip and Enrollment Commissions and the Shifting Boundaries of Belonging, 1885-1920." In <i>Metis and the Medicine Line: Creating a Border and Dividing a People.</i>, 183-226. Chapel Hill: University of North Carolina Press; Regina: University of Regina Press, 2015.</li> <li>Library and Archives Canada, RG 31, Statistics Canada, Census of Canada, 1861, C-1091, District 6, Nipissing, 42-3.</li> <li>Stearne, John. <i>A Confirmation and Discovery of Witches. 1648.</i> Reprint, Devon: The Rota, 1973. Internet Archive. <https://archive.org/stream/confirmationdisc00steauoft/confirmationdisc00steauoft_djvu.txt></li> <li>Uszkalo, Kirsten C. "Brimstone" <i>The Witches in Early Modern England Project.</i> 2011. 12 March 2016. ``<http://witching.org/brimstone>``.</li> --- ### Clipart & Sound Effects <a href="https://github.com/rhymeswithfloral/hist5702w_project/blob/master/Resource_Reference.md">Media and graphic sources</a>.

Double-click this passage to edit it.