Hackathon+ · 23–24 Nov 2019 · Warsaw
see4me — an app project for blind and visually impaired people, built in 24 hours
Hackathon+ 2019 · a team of 6 · 3rd place, awarded by the jury · my role: UX Design
Hackathon+
A 24-hour brief: more independence
“The goal of Hackathon+ was to develop, in 24 hours, a solution that improves the quality and independence of life for people with special needs, including the elderly and people with permanent or temporary mobility or perception difficulties. See4me aims to improve daily life for blind and visually impaired people.”
- When
- 23–24 November 2019project description
- Where
- Warsaw, PGE Narodowy Stadiumproject descriptionPolskie Radio 24
- Organizer
- Ministry of Funds and Regional Policyproject descriptionThe project description also gives the ministry’s earlier name: Ministry of Investment and Development.
- Edition
- secondPolskie Radio 24
- Teams in the competition
- 22Polskie Radio 24
- Participants
- more than 100MFiPR tweet, 23 Nov 2019
- see4me’s result
- 3rd place, awarded by the juryproject description
- Team
- 6 peopleteam list
- My role
- UX Designthe screen flow and the mock-ups’ layout, the app’s content and architecture, the pitch deck and presenting it to the juryaccording to the author
Three examples from the project description
“Among other things, it helps find a bench in a park on a long walk, announces pedestrian crossings along the way and warns about obstacles.”
The project description puts these in the present tense, but per slide 9 only object recognition and text to speech were implemented — all three were planned (chapter 05).
The app’s interface is in Polish.
- plan from the pitch
A park bench
“…find a bench in a park on a long walk…”
project descriptionThe mock-ups have no screen with a bench — neither slide 6 nor the thumbnail.
- plan from the pitch
Pedestrian crossings along the way
Przejścia dla pieszychWłączoneclosest in the mock-upsa category in the settings (mock-up)
- plan from the pitch
Obstacles
Przeszkodyclosest in the mock-upsa category in the settings (mock-up)
Hi-fidelity mock-ups
Five hi-fi mock-ups and the splash screen — to click through
The labels, colors and layout come from the mock-ups. Each transition between screens says how we know it: from the button’s label or from the author; one transition (the app starting up) is our reconstruction.
see4me
The app’s interface is in Polish. See the glossary
Compare works for the slide-6 screens — the thumbnail has different proportions.
Transitions between screens
Pick a transition: the replica opens the screen, outlines the button with a detector box and moves on.
Try it
Done: 0 of 3
- Find the “Praca” route (Work), to doTwoje trasy, the search field in the header
- Add a route of your own, to doNowa trasa, a name, Zapisz trasę
- Switch “Przystanki” (Stops), to dothe Personalizacja screen
Categories in the replicaSwitched on: 2 of 5
Mock-up details
- Two large targets fill the home screen: the top half is “Wybierz trasę” (Choose a route), the bottom half “Nowa trasa” (New route).
- Each state is a symbol and a word: Włączone (On), Wyłączone (Off).
- One sample route starts at PGE Narodowy — where, according to Polskie Radio 24, Hackathon+ took place.Polskie Radio 24
- “Prowadź mnie tam” (Take me there) is a guide-dog icon with a caption under it.
- The microphone on Dodaj trasę (Add a route): voice control was a plan.slide 5
How the replica differs from the mock-ups
- The home screen’s header: “see4mee” on the mock-up, “see4me” in the replica (spelling corrected).
- Slide 6 writes “bankomaty” in lowercase; the replica has “Bankomaty”, as on the thumbnail.
- The fifth row, “Przeszkody” (Obstacles), comes from the thumbnail. The “Wyłączone” (Off) under its circle is a reconstruction: on the thumbnail the bottom bar covers it.reconstruction
- The icons are redrawn, and the labels are set in the device’s system sans-serif (Helvetica Neue, Arial or Liberation Sans), so the typeface may differ slightly from the mock-ups.
- The camera window plays an excerpt of the hackathon recording instead of the mock-up’s still frame.
- The microphone records nothing, and the page never asks for camera or microphone access.
- How the search works (the field itself is on the mock-up), the empty-name check, the new row tagged “added in the replica” and the labels “wznów wykrywanie” (resume detecting) and “pauza” (paused) are ours.reconstruction
Glossary: the app’s Polish labels
- Personalizacja
- Personalization
- Sygnalizacja świetlna
- Traffic lights
- Przejścia dla pieszych
- Pedestrian crossings
- Przystanki
- Stops
- Bankomaty
- ATMs
- Przeszkody
- Obstacles
- Włączone / Wyłączone
- On / Off
- Wybierz trasę
- Choose a route
- Nowa trasa
- New route
- Twoje trasy
- Your routes
- Znajdź trasę…
- Find a route…
- Z: / Do:
- From: / To:
- Dom, Praca, Spożywczak
- Home, Work, Grocery store
- punkty, punktów
- points
- Prowadź mnie tam
- Take me there
- Dodaj trasę
- Add a route
- Nazwij trasę…
- Name the route…
- Zapisz trasę
- Save the route
- Wykrywanie
- Detection
- wykrywam
- detecting
- zatrzymaj wykrywanie
- stop detecting
- wznów wykrywanie, pauza
- resume detecting, paused (our labels)
- Wstecz
- Back
Detection
The camera image, the boxes and the green path
Among the mock-ups, this is the one screen with object recognition — a feature slide 9 lists as implemented. According to the author, the demo is a screen recording of the working prototype from Hackathon+ 2019, and the phone computed the boxes and the path live.
The project description calls it “AR navigation”. On screen, that means boxes and a trapezoid laid over the camera image. The boxes surround a person, a bicycle and deck chairs, each captioned with an English class name and a number, such as “bicycle: 0.9823”. The green trapezoid at the bottom of the image reads as a path, although no mock-up labels it.
Slide 8 lists TensorFlow among the technologies, but no material links it to these boxes or names the model that computed them.
Wykrywanie
Stops the camera window’s recording and the strip’s dashes — like the phone’s own “zatrzymaj wykrywanie” button.
The camera window loops an excerpt of the demo recording (about 3.6 s, flash-free) — the mock-up has a still frame here. The back arrow and the bottom bar take you up to the full replica in chapter 02.
Two frames, label by label
Frame B comes from the project thumbnail, frame A from slide 6. Both are moments of the same demo recording: B is its first frame, and A shows the same scores as the recording about 3.1 s in, framed slightly differently. The labels are transcribed verbatim; “…” marks text cut off by the frame or covered by another label.
Labels on frame B
Pick a label to outline its box on the frame.
Labels on frame A
Pick a label to outline its box on the frame.
Read aloud by your browser’s speech synthesizer — added by this page. The demo recording is silent, so it doesn’t preserve what the app sounded like. It reads only the certain, whole words and numbers, in English, and skips anything cut off.
The demo recording from Hackathon+ 2019
According to the author, this is a screen recording of the working prototype, and the phone computed the boxes and the path live. It runs 12.6 s and is silent (the original file is silent too). The path is green for most of the recording; at times its far part, occasionally all of it, turns red. From about 10.3 s there is no path. The materials don’t say what the red means.
Why
According to the pitch: 1.8 million blind and visually impaired people in Poland
At the start of the pitch: two figures from its Statistics slide, followed by the project’s mission and goal.
Statistics
- 1.8 million
- blind and visually impaired people live in Poland
- 284 million
- blind and visually impaired people live worldwide
Figures from the Statistics slide of the 2019 pitch; the slide does not cite a source.
Improving the safety, comfort and quality of life of blind and visually impaired people.
Helping blind and visually impaired people get around the city and plan their journeys
The slides are in Polish; the quotations are translated.
Planned and built
What was planned and what was implemented
Slide 5 lists eight planned features. Slide 9 marks only two things as implemented: text to speech from this list, and object recognition, which slide 5 does not list. The switch above the list shows both.
Slide 5’s items, set like the mock-ups’ “Personalizacja” list
Slide 5: all 8 items are planned.
Features
- Collision detectionplan, slide 5
- Finding POIsplan, slide 5
- Describing the surroundingsplan, slide 5
- Saving routesplan, slide 5
Navigating the app
- voice controlplan, slide 5
- use without sightplan, slide 5no screen in the mock-ups
How the app communicates
- vibration — phone/
smartwatchplan, slide 5no screen in the mock-ups - text to speechplan, slide 5no screen in the mock-ups
Items and groups from slide 5, translated from Polish. “Closest in the mock-ups” only points to a similar mock-up screen; it says nothing about what worked.
Implementation status
Progress
Pre—MVP
measured on the slide image — the slide gives no number
what has been implemented
- Object recognitionsee the detection chapter
- Text to speechsee the detection chapter
data — declarations from the pitch
- “We collect personal data”
- “We collect data to train our algorithm”
- “We collect data on how users move, to develop the algorithm and future features”
The materials don’t say whether any data was collected, or what kind.
Transcribed from slide 9 and translated from Polish.
Visual system
Yellow on black: a contrast of 19.55 : 1
Colors sampled from the files, computed contrast ratios, a measured row height and the typefaces of the deck and the app. The contrast, palette, typeface, status bar and row tiles are working tools.
Contrast
Preset pairs
The thresholds come from WCAG 2.2 (success criteria 1.4.3, 1.4.6 and 1.4.11): reference values, not an audit of the mock-ups. Large text: from 24 px, or from 18.66 px in bold.
The path on yellow reaches only 1.18 : 1, which is why the green path appears only on black on this page.
Palette
Pixel medians from JPEG files, which add noise. The thumbnail’s background is #FFF466; slide 6 draws the path in #4AFB00.
Status bar
“A11Y” stands where the carrier’s name goes in the status bar — on all five mock-ups of slide 6 and on the thumbnail’s Personalizacja screen. It is the industry’s shorthand for accessibility: the first letter, the 11 letters in between and the last one. The mock-ups don’t say how the shorthand got there.
A list row
A list row on the mock-up: 106 px on a 317 px screen, which is 125 pt at a width of 375 pt (a conversion). For comparison: 44 × 44 px — WCAG 2.5.5 (AAA).
Measured on the mock-up, not on a working app. The row drawn here is at the replica’s scale.
Typeface
Exo 2
Funkcjonalności
The deck’s accent
Znajdowanie POI
Slide 5 (“finding POIs”). The deck uses italics to single out terms and figures — the same goes for “freemium”, “9.99 zł” and “wearables” on slides 11 and 13.
Weights on the slides
- labels and statementsSemiBold 600
- sub-labelsRegular 400
- accentsItalic 400
Measured from the stroke widths on slides 8 and 10–13.
The mock-ups’ sans-serif
Sygnalizacja świetlna
The mock-ups on slide 6 and the thumbnail are set in a system sans-serif. The replica uses the sans-serif available on the device: Helvetica Neue, Arial or Liberation Sans.
Icons
- Sygnalizacja świetlnatraffic lights
- Przejścia dla pieszychpedestrian crossings
- Przystankistops
- BankomatyATMs
- Przeszkodyobstacles
- Personalizacjapersonalization
- Wybierz trasęchoose a route
- Nowa trasanew route
- Znajdź trasę…find a route…
- 6 punktów6 points
- Prowadź mnie tamtake me there
- Dodaj trasęadd a route
- Włączoneon
- Wyłączoneoff
The icons’ meaning and placement come from the mock-ups (slide 6, the thumbnail), and the captions are the mock-ups’ own labels. The shapes are redrawn on a 24-unit grid — none is cut from an image.
All color pairs: 13 — 10 pass their threshold, 3 do not.
| Pair | Where | Contrast | Threshold |
|---|---|---|---|
| From the mock-ups and slides | |||
| #000000 on #FFF467text on the slides | text on the slides | 18.39 : 1 | passesText 4.5 : 1 |
| #000000 on #FFFFFFtext on slide 8 and in slide 14’s Post—MVP zone | text on slide 8 and in slide 14’s Post—MVP zone | 21.00 : 1 | passesText 4.5 : 1 |
| #000000 on #FFF572labels on the app’s bars (slide 6) | labels on the app’s bars (slide 6) | 18.56 : 1 | passesText 4.5 : 1 |
| #FEFF29 on #000000the app’s text (slide 6) | the app’s text (slide 6) | 19.55 : 1 | passesText 4.5 : 1 |
| #FAEA25 on #000000the app’s icons (slide 6) | the app’s icons (slide 6) | 16.86 : 1 | passesUI components 3 : 1 |
| #9B9646 on #000000list rules (slide 6) — not text | list rules (slide 6) — not text | 6.84 : 1 | passesUI components 3 : 1 |
| #393D42 on #FFF467the phone frame on slide 6’s background | the phone frame on slide 6’s background | 9.58 : 1 | passesUI components 3 : 1 |
| On this page | |||
| #42FF0F on #000000the path — on black only | the path — on black only | 15.54 : 1 | passesUI components 3 : 1 |
| #000000 on #FEFF29the knockout accent on black bands | the knockout accent on black bands | 19.55 : 1 | passesText 4.5 : 1 |
| #8F8C1F on #000000dimmed rows (a chosen shade, not a sample) | dimmed rows (a chosen shade, not a sample) | 5.94 : 1 | passesText 4.5 : 1 |
| Never used | |||
| #42FF0F on #FFF467the path on yellow | the path on yellow | 1.18 : 1 | failsUI components 3 : 1 |
| #42FF0F on #FFFFFFthe path on white | the path on white | 1.35 : 1 | failsUI components 3 : 1 |
| #FEFF29 on #FFF467the app’s text on yellow — why the page title never leaves the black beam | the app’s text on yellow — why the page title never leaves the black beam | 1.06 : 1 | failsText 4.5 : 1 |
Ratios computed with the WCAG 2.x formula from the pixel medians. “Passes” refers to the threshold in the “Threshold” column.
The pitch
The pitch’s plan: half a year to the first MVP
The business part of the pitch — the project’s parts and technologies, rollout, monetization, licensing and costs — as the team presented it. Everything here is a plan.
Each step is one slide: its title, its full text (translated from Polish) and what the slide doesn’t say. The card re-typesets it in Exo 2 on the deck’s grid, in English; the “Original” switch shows the slide itself, in Polish.
The parts of the see4me project
The slide doesn’t say which technologies went into the prototype, or into which part.
The technologies are given as names; their logos appear only in the original slide.
The slide’s text
- Mobile app
- Cloud computing
- A website describing the project
technologies
- Flutter
- Go
- Firebase
- Google Cloud
- Bazel
- TensorFlow
- Dialogflow
- Google Maps Platform
Transcribed from slide 8 and translated from Polish.
Rollout method and timeline
The slide doesn’t say when the half year would start.
The slide’s text
- Expected delivery time: half a year for the first MVP
- Using the app will not require any additional software
Transcribed from slide 10 and translated from Polish.
Monetization
The slide doesn’t say how many months would be free.
The slide’s text
Subscription model
- first months free — freemium
- then a subscription of PLN 9.99the slide’s note beside it: “dictated by how little it is felt over a month”
working with the government
- cost refunds as part of improving living conditions for people soon after sight loss / as sight gradually worsens
Transcribed from slide 11 and translated from Polish.
Licensing
The slide doesn’t name the foreign markets.
The slide’s text
we want to expand into foreign markets after first trials in the test market — Poland
Transcribed from slide 12 and translated from Polish.
Estimated rollout costs
One total, not broken down by item.
The slide’s text
PLN 1 million
costs
- User Driven Development
- Target user group research
- Market research
- Product testing
- Product iterations, MVP to MLP
- Buying wearables for testing.
fixed costs
- Paying for cloud computing
Transcribed from slide 13 and translated from Polish.
Schedule
The spending schedule: nine items
Slide 14 as a timeline: six items in the “Pre—MVP release” phase and three in “Post—MVP release”. No dates or amounts — the slide gives none.
Each item on the timeline is a button that opens its description (Enter or Space); the arrow keys also move between the items, and Home and End jump to the first and the last.
Pre—MVP release
The slide gives only the title.
Server costs, the cost of Google queries
The slide gives only the title.
The slide gives only the title.
Università Bocconi, University of Warsaw, University of Wrocław
a spending plan — not an account of consultations that took place
Aligning the rules for processing data with EU requirements
Post—MVP release
Paying for meetings with users and gathering feedback through the whole development process
Buying or renting phones of different brands to test the app
The text under this item repeats item 7.
The pitch’s spending plan — no dates or amounts. Under item 9 the slide repeats the description of item 7, so we show the title only. Translated from Polish.
Press
Hackathon+ on Polskie Radio 24
The Polskie Radio 24 report (24 Nov 2019) and the ministry’s tweet (23 Nov 2019) describe the hackathon where see4me was made.
Markers
Pick a fact — a box marks it on the clipping and its sentence lights up in the transcript.
Polskie Radio 24, 24 Nov 2019
MFiPR tweet, 23 Nov 2019
Transcript
The article
The article and the tweet are in Polish; below is our translation.
- last updated: 24 Nov 2019, 21:13
- Apps to help people with disabilities. Hackaton+ winners announced (original spelling)
- IAR
- The second edition of Hackathon+ has come to an end in Warsaw.
- During the 24-hour marathon at the PGE Narodowy Stadium, programmers from all over Poland looked for solutions that make it easier for people with disabilities and seniors to get around — among other things, in virtual space.
- […] (part of the page left out)
- 22 teams competed for the win.
- The jurors stressed that this year’s edition was of a very high standard and that choosing the winner was not easy.
Transcript
The tweet
- Apps for seniors, blind and visually impaired people, deaf and hard-of-hearing people, and for those with mobility problems — more than 100 #Hackathon+ participants are working on this.
- The programmers aren’t giving away any details yet [two emoji: a face without a mouth and a wink]
In the part of the article on the screenshot, see4me is not named; the project description gives its 3rd place. We do not identify anyone in the tweet’s photos.
Crops of a page screenshot: from the headline to the lead, the paragraph about the 22 teams and the embedded tweet. We left out the rest of the page.
Archive
All the materials and the team
All slides in the materials (as images and as text), the thumbnail, the recording and the press clipping — then the team, sources and rights.
When the slide has focus, the left and right arrow keys change the slide; Home and End go to the first and the last.
- 7 — not in the materials
Other materials
The team
team list- UX Design: Jakub Nowakmy role
- screen flow and mock-up layout
- the app’s content and architecture
- the pitch deck and presenting it to the jury
- Visual Design: Klaudiusz Kędziora
- Data Science: Aleksander Kuśmierczyk
- Data Science: Elton Arreza
- Front-end: Kamil Pilawka
- Back-end: Aleksander Kubych
Sources and rights
Sources
Rights
- slide 3 — a DC trademark, not shown
- the technology names and logos on slide 8 belong to their owners
- the article screenshot (Polskie Radio 24 / IAR) and the tweet with its photos (Ministry of Funds and Regional Policy, @MFIPR_GOV_PL) belong to their owners
- Exo 2 — SIL Open Font License
Notes
- The data in the replica are samples — from the mock-ups.
- Voice: the browser’s speech synthesizer, added by this page.