Document Accessibility

---

Voicing Diagrams and Sonification

Overview

• Wrap up math: Born accessible
• Diagrams:
  • Vector graphics
  • Tactile, audio-tactile, audio
  • Diverse sources

Audio Tactile Graphics

• Combination of tactile graphics and voice output
• ViewPlus IVEO products

Example

Screen Reading

Similar to the Math cases: Important is semantics

Embedding semantics:

• Grouping
• Title and description
• Navigation options
  • z-axis navigation
  • Tabbing navigation
  • Fully navigation via shadow DOM

SVG with information

Source: Highcharts

Arrangement vs Taborder

Order can be determined via arranging visual components

• Breadth First
• Depth First

Or as usual via taborder

• Taborder example

Navigation via Shadow DOM

Generating Semantics

Generation of Semantics

• Manual input
• Image analysis
• From models or representation formats
• Compute from scratch

Diagrams in Alternative Formats

• Providing Alternative text is generally not enough
• Conveying information of most diagrams in a Tweet is nearly impossible
• Automatic recognition works pretty well for "information free" images

Try Automatic Captioning

• Caption Bot
• Cats vs Diagrams

This is often not very useful

Semantics from Computation

• Generating application usually has most information needed
• Unfortunatly SVG export often "stupid"
• Computational models or internal file formats can be exploited

Examples:

• Statistics
  • Models from the R system
  • Diagrams generated with BrailleR
• Structured files examples:
  • Chemistry: mol, cml, etc.
  • Circuit file file formats
  • ...

Statistics Diagram Example

Get Semantics from Scratch

• Image analysis and recognition
• Alternatively: Unique Identifiers
• Generation of annotated SVG
• Semantic enrichment
  • Recursively identify major components
  • Use web services for naming
• Generate hierarchical navigation structure

Chemistry Example

Other things

3D printing

Accessible Documents with Diagrams

Examples:

• These slides
• Diagram Report
• Dedicons Gold Standard for Chemistry

Authoring Diagrams

There are too many ways:

• Make sure to export in a good format, e.g., SVG
• Embed as much information as possible
• Make data files available

Some specific ideas for LaTeX

A tex4ht Workflow

• Source: LaTeX
• Target: HTML

tex4ht and make4ht are part of standard TeX distributions

htlatex filename.tex "xhtml,html5,charset=utf-8" " -cmozhtf -utf8"

Equivalent and simpler:

make4ht foo.tex

To enable mathjax:

make4ht foo.tex "mathjax"

Optional TeX drivers, e.g. \def\pgfsysdriver{pgfsys-dvisvgm4ht.def} for tikz

Graphics

LaTeX graphics packages:

• tikz: tex4ht has decent support
• xyPic: MathJax extension for rendering
• pstricks: best to be avoided

xypic Example

\documentclass{article}
\usepackage[color,matrix,arrow,2cell,all, rotate,pdf]{xy}
\usepackage{xcolor}

\begin{document}
  Consider the following diagram:
  \[
  \xymatrix{
  & {K} \ar@<0.6ex>[d]^{\kappa_2} \ar@<-0.6ex>[d]_{\kappa_1} \\
  {\cdot} \ar@[red]@{.>}[ur]^{\color{red}!~(u, v)} \ar@<0.6ex>[r]^u
  \ar@<-0.6ex>[r]_v & {A} \ar[r]^f \ar[dr]_g & {B}
  \ar@[brown]@{-->}[d]^{\color{brown}!~ h}\\
  & & {\cdot}
  },
  \]
\end{document}

Output with MathJax

Good Practice: Diagram authoring

You have an infinite canvas - use it wisely.

• spread them out
  • e.g., do partial commutative diagrams
• Explain them in text
  • interwoven
  • in proximity
• LaTeX packages
  • tikz: tex4ht has decent support
  • xyPic: MathJax extension for rendering
  • pstricks: best to be avoided

Intermission

Sonification

• Human's have limited capacity (or channels) for speech understanding
  • Try following two conversations at the same time
  • Listen to an audiobook while reading
• Additional information can be conveyed by sound
• Sounds to enhance or replace visual information
• Sounds to replace speech
• Sonification to informative

Sound Enhancement for Speech

Nonspeech sounds to convey information or aid navigation

• Auditory icons
  • Representative real sounds of objects or events
  • Examples: Sending of email, crumbling paper for emptying trash
• Earcons
  • Brief synthesised or musical melody patterns
  • Examples: Error beep, Windows startup sound
• Spearcons
  • (Extremely) speeded up spoken text
  • Examples: Link, image announcements

Note, these definitions vary in the literature!

Datadriven Sound

• Sounds that are generated from data information
• Sounds that try to convey the meaning of data

Use in STEM Accessibility

• Data sonification
• Auditory graphs
• Information layering
  • Multi frequency
  • Multi channel
  • Soundscapes

Methods of Sonification

• Scanning Methods
  • Image data is sonified in a predefined sequence
• Probing Methods
  • User can interactively choose what portion of image is sonified
• Windowing techniques
  • Probing via restricted scanning

Examples

• Bruce Walker's Diagram report

Others

• SAS Graphics Accelerator: Data table visualisation and sonification
• ViewPlus Accessible Graphing Calculator: Probing via tactile tracing
• Audio Functions: Scanning, Probing and Speech
• graCALC: Overview Sonification and Speech Description

Web Examples

• SAS Graphics Accelerator
• Highcharts
  • Uses SVG
• Desmos
  • Uses Canvas drawing
• EquatIO
  • Uses Desmos for graphs

Technical Realisation

Using the Web Audio API

• Simple Oscillator, e.g.,

    let context = new AudioContext();
    let oscillator = context.createOscillator();

• Oscillator represents an audio source that generates a periodic waveform (e.g., a sine wave) for a given frequency
• It allows to turn data values into frequencies over time

Scanning Method

• Let $P = {(x_0,y_0), \ldots, (x_n,y_n)}$ be a path given by set of $(x_i,y_i)$-coordinate pairs.
• Let $\sigma$ is a suitable scaling function.

We define a base frequency $f_0=\sigma(y_0)$. Then map all remaining $y$ coordinates to frequencies $f_i=\sigma(y_i)$.

• Let $\tau$ be a another scaling function.
• Define $n$ time intervals $[t_{i-1}, t_i], i = 1, \ldots, n$ by $t_i = \tau(x_i)$.

We can then create $n$ oscillators $o_i, i=1,...,n$ with frequency $f_i$.

A sonification is then created by running oscillator $o_i$ in interval $[t_{i-1}, t_i]$.

Results

This approach gives us a scanning method

• The created sound will be rather crude.
• Apply smoothing for more pleasing results
• Or modulate externally loaded sounds instead of using oscillators

Probing Method

Probing can be achieved easily by

• combine scanning with interactive exploration
• allow reader to through recursively decreasing parts of the data plot
• add a sonification to each sub-interval separately

Example

• Boxplot
• Timeseries

Outlook

• Advanced Content
• Interactivity
• Research Questions