Author SHA1 Message Date
Seb3thehacker e8f51f4947 0.3.01-alpha: wireless around VPNs and early Wi-Fi details, re-signed apps lose the red X 2026-10-15 12:38:06 -07:00
Seb3thehacker 2281347728 links: the repo now lives at twinksoftware/gearslip 2026-10-15 12:38:06 -07:00
Seb3thehacker 6d08fbd871 license: Gearslip now belongs to Twink Software LLC 2026-10-15 12:38:06 -07:00
Seb3thehacker 1322cbaf39 readme: package name and certificate hash for AppVerifier 2026-10-15 08:04:28 -07:00
Seb3thehacker db6f1b8e82 launcher: a patched Waze, signed with someone else's key, loses the red X and can start as the map app 2026-10-15 07:50:34 -07:00
Seb3thehacker 0177975bc8 wireless: wait for the address when the Wi-Fi details come first, and work around or name a VPN that blocks the car 2026-10-15 07:25:59 -07:00
Seb3thehacker 5f30e7b014 readme: an eighth screenshot, coffee nearby in Vela 2026-10-15 07:21:47 -07:00
Seb3thehacker c42406f353 support: Liberapay first again, then Buy Me a Coffee, Monero last 2026-10-14 23:41:34 -07:00
Seb3thehacker e8d025df47 support: say Buy Me a Coffee keeps 5% 2026-10-14 23:39:05 -07:00
Seb3thehacker c6d8d8e787 support: Buy Me a Coffee first in Settings, the popup, the thank-you page and the repo's Sponsor button; Monero last 2026-10-14 23:36:30 -07:00
Seb3thehacker 77b84212db readme: new screenshots for 0.3, with the assistant, a route in the bar, and the song title 2026-10-14 23:34:35 -07:00
Seb3thehacker b12e9651c1 wireless: when Android won't ask for the permissions, a button to enable Nearby devices in the app's settings 2026-10-14 22:06:24 -07:00
Seb3thehacker a5c46967b7 voice: drop the unused archive download and commons-compress with it, which CodeQL flagged 2026-10-14 21:11:40 -07:00
Seb3thehacker 30a4b65a08 release: notes list calls as a known issue, and new releases start as pre-releases 2026-10-14 19:57:32 -07:00
Seb3thehacker 88ff5b0606 voice: Parakeet small downloads as plain files, like the other models, instead of an archive the phone unpacks slowly 2026-10-14 19:56:34 -07:00
Seb3thehacker aa15884897 Merge branch 'experimental' 2026-10-14 19:49:37 -07:00
Seb3thehacker 84e1145459 intro: drop the Let's drive page, so it ends on thanks 2026-10-14 19:45:47 -07:00
Seb3thehacker 53de4a2512 0.3.00-alpha, with its notes in the app and on the GitHub release 2026-10-14 19:40:25 -07:00
Seb3thehacker ed46868d9b voice: say Canary knows English, Spanish, German and French, and that Gearslip listens in English for now 2026-10-14 19:36:23 -07:00
Seb3thehacker 2bf4753125 support: Monero, in a popup with the full address, Open in wallet and Copy address 2026-10-14 19:29:57 -07:00
Seb3thehacker 879b428d1f support: a Liberapay button on the intro's thank-you page, no "through a nonprofit", and room for Monero and a give-once card link, hidden until they're set 2026-10-14 19:21:41 -07:00
Seb3thehacker 35bd239c2f support: Liberapay only until GitHub Sponsors is approved, and "support" instead of "donate", since giving to the LLC isn't tax-deductible 2026-10-14 19:17:19 -07:00
Seb3thehacker 815a75ad6d car: say when the phone's location is off, ten seconds into a drive, with Don't show again and a setting to turn the warning off 2026-10-14 15:45:10 -07:00
Seb3thehacker 9b7734bec8 voice: "navigate to X" picks the nearest X in the map app's own search and stops at its start screen; it no longer sends the phone's position, which Organic Maps took as the destination 2026-10-14 15:45:10 -07:00
Seb3thehacker 7ffa203221 voice: a new destination mid-route ends the old route first instead of being ignored; switching or closing map apps ends the old app's route 2026-10-14 14:24:47 -07:00
Seb3thehacker 9f982875fe release builds shrink with R8: 65 MB down to 30 MB, small enough for IzzyOnDroid. -Pr8 does the same to a debug build for testing 2026-10-14 13:29:00 -07:00
Seb3thehacker 5c799fed38 Change Liberapay entry to username only
Updated Liberapay username to remove URL.
2026-10-14 12:53:27 -07:00
Seb3thehacker 249e6cefed Add Liberapay username to FUNDING.yml
Updated Liberapay entry with a specific username.
2026-10-14 12:51:45 -07:00
Seb3thehacker 3c9603175e debug builds send usage notes too, marked debug so the server keeps them out of the stats; call them debug builds everywhere instead of dev 2026-10-14 12:25:45 -07:00
Seb3thehacker 10dfb46a96 player: draw the app's own like, shuffle and repeat buttons, lit when their icon says they're on (Tidal showed every song liked and shuffle off) 2026-10-14 00:42:04 -07:00
Seb3thehacker e587ea97e1 intro: tell updates from new installs by setup having been done, so clearing data counts as new 2026-10-13 19:11:33 -07:00
Seb3thehacker daa6f24289 intro: new page order ending on thanks, a plainer welcome, and it resumes where it was after the app restarts 2026-10-13 19:08:48 -07:00
Seb3thehacker 76955669f5 intro: only after an update, never on a new install; recommend Organic Maps before Vela and ViVi Music before Metrolist; the thanks page links the list of cars that work 2026-10-13 19:00:51 -07:00
Seb3thehacker cd7b13c88e intro: Back and Next buttons like the setup guide, instead of tapping anywhere 2026-10-13 18:15:26 -07:00
Seb3thehacker 3d642c17c7 home: the Bluetooth warning is yellow, not red 2026-10-13 18:12:42 -07:00
Seb3thehacker cce3b50b52 music: an app whose library never answers falls back to its playback controls after 8 seconds, instead of spinning for good 2026-10-13 18:09:04 -07:00
Seb3thehacker 88e1885b2d car audio: with Bluetooth audio chosen, warn on the car and on Home when the phone isn't connected to the car 20 seconds in 2026-10-13 18:09:04 -07:00
Seb3thehacker a84d36f17a voice: app names match however the recognizer spaces them, so "with map quest" picks MapQuest 2026-10-13 17:57:30 -07:00
Seb3thehacker 8b269f4734 voice: opening a screen mid-download no longer deletes the download in progress 2026-10-13 17:54:36 -07:00
Seb3thehacker 3042e01e75 donations: at the top of Settings, and a popup on Home after the third drive that worked, then at most every three months 2026-10-13 17:42:05 -07:00
Seb3thehacker 71397a0777 settings: donation links are back, in their own section above About 2026-10-13 17:39:15 -07:00
Seb3thehacker cae05ecdc0 car launcher: only apps that don't work get a mark, a red X, and the badge key is gone 2026-10-13 17:39:15 -07:00
Seb3thehacker 73be554eb2 car audio: pause the phone's music when the car switches to the radio, send AAC to cars that take nothing else, and open GM's own channel in GM cars 2026-10-13 17:14:32 -07:00
Seb3thehacker 2812613bca wireless: say the car must be paired first, with a button to Bluetooth settings 2026-10-13 17:14:32 -07:00
Seb3thehacker 014942338b usage notes: shorter intro and options, so all four fit on screen 2026-10-13 17:04:47 -07:00
Seb3thehacker 3c81980385 usage notes: name every field each option sends 2026-10-13 16:58:46 -07:00
Seb3thehacker c029a0894a voice: one button downloads Kroko and Canary in setup, with progress on Home and in a silent notification 2026-10-13 16:56:56 -07:00
Seb3thehacker eba8ac1127 home: What's new and which cars work sit at the bottom 2026-10-13 16:45:59 -07:00
Seb3thehacker 1086e7c8fa cars: a list of cars drivers have confirmed, linked from the README and the help screen 2026-10-13 16:40:21 -07:00
Seb3thehacker 470ec2f093 usage notes: say whether each drive was USB or wireless, and whether music went over USB or Bluetooth 2026-10-13 16:24:28 -07:00
Seb3thehacker 341e62d1d8 readme: notes say which boxes are ticked 2026-10-13 16:17:49 -07:00
Seb3thehacker 2891a170db usage notes: say which boxes are ticked, so the server can drop details from unticked ones 2026-10-13 16:17:49 -07:00
Seb3thehacker 3855cc4781 readme: wireless, voice, the Bluetooth or USB audio choice, and updates 2026-10-13 16:13:02 -07:00
Seb3thehacker f1a6901fbc usage notes: ask after the first good drive only if setup never asked 2026-10-13 16:11:43 -07:00
Seb3thehacker b848c5d910 home: What's new and which cars work under the status card; the usage notes switch stays in Settings 2026-10-13 16:06:26 -07:00
Seb3thehacker 4f1e6dcbfa wireless: Disconnect works while the car hasn't answered yet 2026-10-13 16:00:40 -07:00
Seb3thehacker 10e8568aba intro: thank note senders for confirming cars newer than 2020 2026-10-13 15:55:29 -07:00
Seb3thehacker 724dcf25c7 home: put Connect wirelessly in the status card, with the usage notes switch always under it 2026-10-13 15:48:54 -07:00
Seb3thehacker dd1fd0242f home: put Connect wirelessly in the status card, as a full button 2026-10-13 15:41:59 -07:00
Seb3thehacker 6faad9fdd5 help: say most cars work, group the steps, and show the adb steps only for USB audio 2026-10-13 15:39:20 -07:00
Seb3thehacker 3869258e0e home: show only the car's status; help, preview, logs and the dev test message move to Settings 2026-10-13 15:38:40 -07:00
Seb3thehacker 7d9eb20ff0 updates: hide the channel picker until a pre-release ships 2026-10-13 15:27:15 -07:00
Seb3thehacker 8fd840bd8d updates: offer the final release to people on its beta or alpha 2026-10-13 15:21:17 -07:00
Seb3thehacker 5947bc1e63 intro: add a step inviting drivers to the Matrix chat 2026-10-13 15:21:17 -07:00
Seb3thehacker e13b0618db voice: recommend Canary for the final words, at a third of Parakeet large's size 2026-10-13 15:15:41 -07:00
Seb3thehacker 9100dac290 settings: tidy the order, fold updates into one row with Stable, Beta and Alpha, and drop the donation asks and the after-drive notification 2026-10-13 15:05:45 -07:00
Seb3thehacker f1fe94c487 bluetooth: offer to pair with the car on first connect, and pick Bluetooth or USB audio in setup 2026-10-13 14:55:11 -07:00
Seb3thehacker 5fe59d350c wireless: send about a third of the cable bitrate at 30fps, so the video keeps up and the music stops cutting out 2026-10-13 14:31:03 -07:00
Seb3thehacker 909c3549d6 wireless: ask for Android 17's local network permission, without which the car never hears from the phone 2026-10-13 14:12:13 -07:00
Seb3thehacker 1514721498 wireless: connect over the phone's main Wi-Fi when it's already on the car's network, and log why traffic is held back 2026-10-13 14:03:55 -07:00
Seb3thehacker 4000e0a9e9 wireless: keep trying over the phone's own Wi-Fi when Android drops the car's network 2026-10-13 13:19:22 -07:00
Seb3thehacker 02d6cb546e Merge branch 'experimental' into wireless 2026-10-13 12:41:09 -07:00
Seb3thehacker 59f323ec06 directions: speak each turn on the car's guidance channel
Map apps' own voice prompts can't reach the car over USB, so Gearslip reads
the turn the cluster already gets and says it through the car's directions
sink, which ducks the music. On by default, with a switch in car settings.
2026-10-13 12:26:31 -07:00
Seb3thehacker 0d0bfd3d66 setup: the 0.3 look, with a badge, a big headline and shorter words on every step 2026-10-13 01:28:54 -07:00
Seb3thehacker b3f2088a1d intro: pick the car's button style, with sample buttons that redraw as you choose 2026-10-13 01:14:06 -07:00
Seb3thehacker 01a41c01c8 voice: "pause" keeps the music paused after the mic lets go 2026-10-13 00:52:00 -07:00
Seb3thehacker 62b1835444 theme: flat is the default 2026-10-13 00:45:39 -07:00
Seb3thehacker 1cd3d6f856 car mic: open a voice session and name the codec before asking for the car's microphone 2026-10-13 00:37:49 -07:00
Seb3thehacker 8912e88406 voice: "navigate to the nearest gas station" searches for a gas station near the car 2026-10-13 00:34:49 -07:00
Seb3thehacker 0bdc51ff6c voice: answer back when sworn at 2026-10-13 00:09:58 -07:00
Seb3thehacker 361caf5ea3 car audio: name the codec in the audio Setup, so cars that rejected it play music over USB 2026-10-13 00:09:27 -07:00
Seb3thehacker d53535e56c voice: download the models as plain files, skipping the slow bzip2 unpacking 2026-10-12 23:59:43 -07:00
Seb3thehacker 49a064acc0 video: cars that give no touchscreen size get the largest picture up to 1080p 2026-10-12 23:52:03 -07:00
Seb3thehacker 6d4389c7ab touch: shift taps past the car's cropped margin, so buttons land where they're pressed 2026-10-12 23:50:24 -07:00
Seb3thehacker 9fe463c4e7 Merge branch 'experimental' into wireless
# Conflicts:
#	gearslip/src/main/java/app/seb3thehacker/gearslip/ui/HomeScreen.kt
2026-10-12 22:09:43 -07:00
Seb3thehacker a3761534bb security: scan the Kotlin code with CodeQL by building the app 2026-10-12 21:10:36 -07:00
Seb3thehacker ceed61def9 security: scan the Kotlin code with CodeQL by building the app 2026-10-12 21:10:35 -07:00
Seb3thehacker fd0ef64a42 security: a weekly Trivy scan and Dependabot updates 2026-10-12 21:00:25 -07:00
Seb3thehacker 0d29263f0a security: a weekly Trivy scan and Dependabot updates 2026-10-12 21:00:20 -07:00
Seb3thehacker 28d99a3d87 voice: "play X" reopens the music app used last when none is open or it was closed 2026-10-12 17:16:25 -07:00
Seb3thehacker a526db6cd1 templated search: keep the keyboard down when the app asks, so results opened by voice show 2026-10-12 17:09:51 -07:00
Seb3thehacker 000ca85479 voice: Canary 180M as a middle choice for the final words 2026-10-12 16:38:32 -07:00
Seb3thehacker e76e9ddcf7 navigation: search near the phone, so "navigate to Del Taco" finds the closest one 2026-10-12 16:21:13 -07:00
Seb3thehacker 8069f54d8f voice: free Parakeet after 5 minutes without use and both models when the car disconnects 2026-10-12 16:20:15 -07:00
Seb3thehacker df85b0d1f3 car keyboard: taller keys, and search sits in the bottom-right corner 2026-10-12 16:19:27 -07:00
Seb3thehacker 4ac3d172d3 voice: understand natural phrasing for every command, open Gearslip's own apps by name, and say "exit" to hand the screen back 2026-10-12 16:10:23 -07:00
Seb3thehacker d099193ead car: call the Web app Browser 2026-10-12 16:02:53 -07:00
Seb3thehacker bc8f2372b1 voice: start listening sooner, say so while it gets ready, and scroll the help list from anywhere on it 2026-10-12 16:02:53 -07:00
Seb3thehacker 000bd1f3f4 bigger install image 2026-10-12 15:49:07 -07:00
Seb3thehacker 5fa9ed17e1 remove . 2026-10-12 15:44:21 -07:00
Seb3thehacker e2b9118354 readme: drop the link to the stats server's code 2026-10-12 15:43:21 -07:00
Seb3thehacker e1233cb1e4 readme: how Gearslip is made, a contributing section, and some badges 2026-10-12 15:40:56 -07:00
Seb3thehacker 3ca87d1afc log: explain that answering every car with protocol 1.3 is on purpose 2026-10-12 14:16:51 -07:00
Seb3thehacker 8e98851e47 0.3 intro: a recommended apps screen, and AntennaPod in the recommended list 2026-10-12 14:12:04 -07:00
Seb3thehacker af5308a9f9 readme: say plainly that AI writes nearly all the code 2026-10-12 12:44:26 -07:00
Seb3thehacker 7d6b9cf578 car audio: run the audio threads at audio priority so a busy phone doesn't make it skip 2026-10-12 01:31:24 -07:00
Seb3thehacker b2df84c46b voice and media search are built in now, with a voice intro screen and a new overlay over the car screen 2026-10-12 01:31:24 -07:00
Seb3thehacker 30d94bcfb5 voice: replace Whisper with sherpa-onnx, live words from Kroko and the final words from Parakeet
Models download in Settings instead of shipping in the APK. Credits for the engine and models are in About.
2026-10-12 00:51:31 -07:00
Seb3thehacker af9e9a7571 voice: play songs by name, a help list, and the phone's speech recognition as an option; Spotify connects without the wait 2026-10-12 00:20:36 -07:00
Seb3thehacker 7163598352 car media: drop the "Audio to car" line; name every audio focus state 2026-10-12 00:20:36 -07:00
Seb3thehacker 94ed58e13f 0.3 intro, donation links on Home, and About links for Matrix and the source 2026-10-12 00:20:36 -07:00
Seb3thehacker 59657c4f80 ixon fix mono color 2026-10-11 20:20:02 -07:00
Seb3thehacker 8abc8fe973 remove the "Before you plug in" popup on first launch 2026-10-11 20:01:36 -07:00
Seb3thehacker 4fe9421884 car audio: explain a blocked microphone permission instead of failing silently 2026-10-11 20:01:36 -07:00
Seb3thehacker 04b820cb22 wireless: wait for the car to connect when it publishes no Android Auto service 2026-10-11 17:22:58 -07:00
Seb3thehacker 669bf1f7ce Merge branch 'experimental' into wireless
# Conflicts:
#	gearslip/src/main/java/app/seb3thehacker/gearslip/GearslipActivity.kt
#	gearslip/src/main/java/app/seb3thehacker/gearslip/ui/HomeScreen.kt
2026-10-11 17:22:58 -07:00
Seb3thehacker 83916a6397 wireless connection attempt 2026-10-11 16:10:24 -07:00
Seb3thehacker 5a57608591 matrix link 2026-10-11 16:08:50 -07:00
Seb3thehacker f58ecf303a explain testing process and define AI use 2026-10-11 15:37:57 -07:00
Seb3thehacker 61a7188ad2 Add a contributor license agreement
Contributors keep their copyright and license their work to Twink Software LLC, which keeps it available under the AGPL and may also offer it under the commercial license. CONTRIBUTING now points to it.
2026-10-11 14:48:52 -07:00
Seb3thehacker fff2db63b0 updates: beta channel, and a switch to turn off automatic checks
Beta updates counts GitHub pre-releases as updates. With Check automatically off, Gearslip contacts GitHub only when you tap Check for updates.
2026-10-11 14:48:46 -07:00
Seb3thehacker fb6bc018d5 Add a contributor license agreement
Contributors keep their copyright and license their work to Twink Software LLC, which keeps it available under the AGPL and may also offer it under the commercial license. CONTRIBUTING now points to it.
2026-10-11 14:48:46 -07:00
Seb3thehacker 580a8365eb car screen: stop vibrating the phone on long presses 2026-10-11 11:39:22 -07:00
Seb3thehacker 5721b9efb3 icon: remove unused archive images 2026-10-11 11:34:06 -07:00
Seb3thehacker ed331088cc stats: name the phone's OS instead of only GrapheneOS or other
Reports GrapheneOS, CalyxOS, /e/OS, iodeOS, DivestOS, LineageOS, or stock with the brand.
2026-10-11 11:34:06 -07:00
Seb3thehacker 33ed2d0653 setup: drop the GrapheneOS exploit protection step 2026-10-11 11:34:06 -07:00
Seb3thehacker 261e6a719a car audio: choose Bluetooth or USB, offer Bluetooth after a refused screen share
USB audio needs Android's screen-sharing consent every drive. A new Car audio setting on the phone and car screens picks Bluetooth instead, and the first refusal opens a dialog offering Bluetooth, asking again, or the one-time adb command.
2026-10-11 11:34:06 -07:00
Seb3thehacker f9e38cbc45 icon: drop the black square behind the README icon 2026-10-11 00:59:45 -07:00
Seb3thehacker cfa01d4195 icon: drop the black square behind the README icon 2026-10-11 00:59:36 -07:00
Seb3thehacker e843f2adfe car knob: move focus with the rotary knob, show where it is
The knob's turns reached the car screen but never moved focus: keys handed
straight to the view left it in touch mode, where buttons refuse focus.
Buttons now show an accent ring while the knob is on them, map buttons wake
on a knob turn instead of only on touch, and web pages get the same ring.

The car's Home button opens the dashboard, Map opens the map, and Music
reopens the last media app.

Refs #9
2026-10-10 17:55:36 -07:00
Seb3thehacker 0af16d3348 Merge pull request #16 from iscle/fix/connection-handshake-lifecycle
Stop cleanly and ignore messages after handshake rejection
2026-10-10 14:47:46 -07:00
Iscle 8743e992d7 Stop cleanly and ignore messages after handshake rejection 2026-10-10 23:34:39 +02:00
Seb3thehacker 6f9fc061e3 tell the car about the Wi-Fi join before TCP 2026-10-09 15:30:41 -07:00
Seb3thehacker 239be1256b place pages: name on top, actions as buttons, call a place's number
Panes shaped like a place page (MAPS.ME's) show the name and kind first, short
tappable rows as buttons, and the details compact below. A phone number gets a
Call button that asks first with the dialer's confirmation. Other panes draw as
before.
2026-10-09 15:26:48 -07:00
Seb3thehacker 23f12884da car toasts: take turns instead of replacing each other
A new toast waits until the one on screen has had 2 seconds. Repeats restart the
visible toast, and a burst keeps only the newest three waiting.
2026-10-09 15:26:48 -07:00
Seb3thehacker 75dfaa3013 car icons: drop app tints that can't be seen, mark MAPS.ME and MapQuest as working
MAPS.ME tints its map buttons dark grey for dark mode, which vanished on Gearslip's
dark buttons. A tint below 3:1 contrast now falls back to the normal text colour,
as Android Auto does.
2026-10-09 15:26:48 -07:00
Seb3thehacker 179a51ea25 car toasts: match the message popup and stay up longer
A driver only glances at the screen, so short toasts now last 4 seconds and long
ones 7. The card uses the popup's colours, which read better over a map.
2026-10-09 14:24:32 -07:00
Seb3thehacker 006823232d Merge remote-tracking branch 'origin/experimental' into experimental 2026-10-09 14:24:28 -07:00
Seb3thehacker 0838bbb3cc Merge pull request #11 from iscle/feature/host-toasts
Show app toasts on the projected car display
2026-10-09 14:23:50 -07:00
Seb3thehacker 52cb769adc Merge pull request #10 from iscle/feature/host-media-category
Recognize media templates and group them under Music
2026-10-09 14:18:12 -07:00
Seb3thehacker 9c8b82d66a Merge branch 'experimental' into feature/host-media-category 2026-10-09 14:14:28 -07:00
Seb3thehacker 92d7940f56 car apps: let hosted apps start their own location service, fixing MapQuest
Since Android 14, an app bound by a visible host may start a foreground service
only if the host binds it with BIND_ALLOW_ACTIVITY_STARTS. Without it, MapQuest
crashed as navigation began, and Flitsmeister and Vela ran their services without
location. MapQuest now gets the working check.
2026-10-09 13:50:40 -07:00
Iscle 4c2c8c6dda Show app toasts on the projected car display 2026-10-09 20:58:12 +02:00
Iscle 4a6305052d Recognize media template apps and group them under Music 2026-10-09 20:50:43 +02:00
Seb3thehacker fd658d4e57 certificate: say where it comes from, offer import alongside download, keep cert files out of the repo 2026-10-08 15:27:48 -07:00
Seb3thehacker ddaf3e5657 move the certificate screen changes to the experimental branch 2026-10-08 15:27:36 -07:00
Seb3thehacker a549077c13 certificate: say where it comes from, offer import alongside download, keep cert files out of the repo 2026-10-08 15:25:04 -07:00
Seb3thehacker 246f571a86 readme: around 2020 2026-10-08 10:37:53 -07:00
Seb3thehacker a22d7936d4 readme: move which cars work to the top 2026-10-08 10:37:06 -07:00
Seb3thehacker 74330db01c readme: say most cars work, and why some newer ones don't 2026-10-08 10:35:30 -07:00
Seb3thehacker 516f6c7f94 ask once to share after the first drive that works, cache the cert for settings 2026-10-08 00:27:54 -07:00
Seb3thehacker 40086e061b audio pause when unplugged, theme tweak 2026-10-08 00:06:34 -07:00
Seb3thehacker c212c2e2d1 usage notes changes, audio changes, video quality increase. 2026-10-07 23:50:47 -07:00
Seb3thehacker fb4b0e2914 connection proxy negotiation 2026-10-07 23:41:16 -07:00
Seb3thehacker d0fd6ecb5c wireless crash fix 2026-10-07 22:40:41 -07:00
Seb3thehacker 39e1a5ee09 wireless func added (testing) 2026-10-07 18:45:57 -07:00
Seb3thehacker 55320c4d98 example data 2026-10-07 16:37:40 -07:00
Seb3thehacker 9dc705c8cf direct download link to obtainium 2026-10-07 15:11:06 -07:00
Seb3thehacker be981b0899 start car audio as soon as the head unit answers setup 2026-10-07 00:10:28 -07:00
Seb3thehacker dd3b845c7f stream directions to car directly 2026-10-06 23:33:18 -07:00
Seb3thehacker c57abecc62 send song data to car screens 2026-10-06 23:33:18 -07:00
Seb3thehacker dfebae93b3 higher video bitrate, scaled to the car's screen
800x480 at 30fps gets 8 Mbps instead of 4, and larger screens scale from
there. The encoder now declares the H.264 level each size needs, and the
bitrate stays within that level's limit.
2026-10-06 23:33:18 -07:00
Seb3thehacker 1af5eae8e7 usage notes switch on the Home screen 2026-10-06 23:33:18 -07:00
Seb3thehacker 25b8bbb1a5 keep the stats server out of the app repo 2026-10-06 23:33:18 -07:00
Seb3thehacker 2d5000e313 keep the stats server out of the app repo 2026-10-06 23:25:28 -07:00
Seb3thehacker 17534d5784 update checker: finds new releases, downloads and installs them
Checks GitHub's latest release once a day when the app opens and after a
drive. A new version shows as a card on Home, a notification after the
drive, and a card on the car screen at plug-in while parked. Downloads are
checked against GitHub's SHA-256 and never install while the car is
connected. Update notifications can be turned off in Settings.
2026-10-06 00:06:06 -07:00
126 changed files with 9228 additions and 1607 deletions

No files matched your search

+15
View File
@@ -0,0 +1,15 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: Seb3thehacker
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: seb3thehacker
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
+1 -1
View File
@@ -5,7 +5,7 @@ body:
- type: markdown
attributes:
value: |
Thanks for reporting. Check the [open issues](https://github.com/Seb3thehacker/gearslip/issues) first, in case someone has already reported it.
Thanks for reporting. Check the [open issues](https://github.com/twinksoftware/gearslip/issues) first, in case someone has already reported it.
- type: input
id: car
+15
View File
@@ -0,0 +1,15 @@
# Weekly pull requests for newer Gradle libraries and GitHub Actions, aimed at experimental,
# where unreleased code lands. Security fixes come separately, as soon as an alert opens.
version: 2
updates:
- package-ecosystem: gradle
directory: /
target-branch: experimental
schedule:
interval: weekly
open-pull-requests-limit: 5
- package-ecosystem: github-actions
directory: /
target-branch: experimental
schedule:
interval: weekly
+15
View File
@@ -0,0 +1,15 @@
Gearslip 0.3.01 connects wirelessly on more phones and dongles.
## Fixed
- Wireless works with dongles that send their Wi-Fi details before their address.
- A VPN no longer stops a wireless drive without a word. Gearslip tries the phone's ordinary route around it, and if the VPN still blocks the car, Gearslip says so. To fix it for good, exclude Gearslip in your VPN app.
- When Android won't ask for the Nearby devices permission, a button opens Gearslip's settings, where you can turn it on.
## Changed
- An app that someone else has re-signed, such as a modified Waze, no longer shows the red X when it accepts Gearslip.
- Support Gearslip through Liberapay, Buy Me a Coffee, or Monero, in Settings.
- Gearslip now belongs to Twink Software LLC, and its code lives at github.com/twinksoftware/gearslip. Old links still work.
- The README lists Gearslip's signing certificate, so you can check your copy with AppVerifier.
## Known issues
- Phone calls are unstable right now and may drop or fail to connect.
+50
View File
@@ -0,0 +1,50 @@
name: CodeQL
# Kotlin can only be scanned while it compiles, so this builds the debug app, the same way
# release.yml does but without a signing key. Workflow files need no build.
on:
push:
branches: [main, experimental]
pull_request:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch:
jobs:
analyze:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
actions: read
strategy:
fail-fast: false
matrix:
include:
- language: actions
build-mode: none
- language: java-kotlin
build-mode: manual
steps:
- uses: actions/checkout@v4
- uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- if: matrix.build-mode == 'manual'
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "21"
- if: matrix.build-mode == 'manual'
uses: gradle/actions/setup-gradle@v4
- if: matrix.build-mode == 'manual'
run: ./gradlew :gearslip:assembleDebug --no-daemon
- uses: github/codeql-action/analyze@v4
with:
category: "/language:${{ matrix.language }}"
+5
View File
@@ -45,4 +45,9 @@ jobs:
uses: softprops/action-gh-release@v2
with:
files: ${{ steps.apk.outputs.path }}
# The hand-written notes go first; GitHub adds the full changelog link under them.
body_path: .github/release-notes.md
generate_release_notes: true
# Every release starts as a pre-release: only phones on the Alpha update channel get it.
# Once it's tested, untick pre-release on GitHub and everyone gets it.
prerelease: true
+34
View File
@@ -0,0 +1,34 @@
name: Security scan
# Trivy looks through the repository for committed secrets and risky workflow or build
# settings. Findings show under the Security tab, next to CodeQL's and Dependabot's.
on:
push:
branches: [main, experimental]
pull_request:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch:
jobs:
trivy:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- uses: actions/checkout@v4
- uses: aquasecurity/trivy-action@v0.36.0
with:
scan-type: fs
scanners: vuln,secret,misconfig
format: sarif
output: trivy.sarif
severity: MEDIUM,HIGH,CRITICAL
- uses: github/codeql-action/upload-sarif@v4
if: always()
with:
sarif_file: trivy.sarif
category: trivy
+10 -2
View File
@@ -11,7 +11,7 @@ local.properties
*.keystore
keystore.properties
# Native build cache.
# Native build cache
.cxx/
# Testing
@@ -29,4 +29,12 @@ CLAUDE.md
# Other
archive/
.wrangler/
.wrangler/
stats-worker/
# Certificates and keys never go in the repo; the app downloads or imports them on the phone.
*.p12
*.pfx
*.pem
*.crt
*.key
+49
View File
@@ -0,0 +1,49 @@
# Cars
Drivers who share usage notes tell us which cars Gearslip works in. This list comes from
those notes, and from our own testing. A car shows up here once Gearslip has reached its
screen at least once.
The year is the one the car reports. Some cars report the year of the head unit instead,
and some don't say.
## Works
| Car | Year | Head unit |
|---|---|---|
| Audi | | MIB2+ (Alpine) |
| Audi | | Harman |
| Chevrolet, with myChevrolet | 2019, 2020, 2021 | myChevrolet |
| Chevrolet, with MyLink | 2017 | MyLink |
| Dacia | | Media Nav (LG ULC 4.1) |
| Dacia | | LG ULC 4.5 |
| Ford | 2018 | SYNC 3.0 (Panasonic) |
| Ford Fiesta | 2019 | SYNC 3.2 (Panasonic) |
| Ford Fusion | 2018 | SYNC 3.1 (Panasonic) |
| Jeep Cherokee | 2019 | Uconnect 8.4 (Delphi) |
| MG | 2019 | Desay SV |
| Ram 1500 | 2018 | Uconnect 8.4 (Panasonic) |
| Škoda | 2018 | MIB Standard2 (TechniSat) |
| Subaru Outback | 2020 | Denso |
| Toyota RAV4 | 2020 | Pioneer T1 |
| Volkswagen | 2018 | MIB Standard2 (Delphi) |
| Volkswagen | 2024 | ICAS3 |
| Volvo XC90 | 2022 | SPA Plus (Mitsubishi Electric) |
| Wuling | 2021 | Desay SV |
| [LIVI](https://github.com/f-io/LIVI), the open-source head unit | | |
## Not working yet
| Car | Year | What happens |
|---|---|---|
| Honda | 2016 | The connection drops before Gearslip reaches the screen. |
| Honda | 2019 | The connection drops before Gearslip reaches the screen. |
| Mitsubishi Attrage | 2020 | The connection drops before Gearslip reaches the screen. |
Some cars reject Gearslip's certificate before they say what they are, so they can't appear
here by name. Most of these are from around 2020 or later.
## Add your car
Turn on usage notes in Gearslip's setup or Settings, and drive once. If your car is missing
or listed wrongly, open an issue.
+102
View File
@@ -0,0 +1,102 @@
# Gearslip Contributor License Agreement
Version 1.0, October 2026
Thank you for contributing to Gearslip. This agreement lets Twink Software LLC, the company
behind Gearslip, use your contribution. You keep the copyright in your work. You sign once,
and the agreement covers every contribution you make to Gearslip, past and future.
## Why we ask
Gearslip is free software under the GNU Affero General Public License, version 3 (the
"AGPL"). Twink Software LLC also sells commercial licenses to companies that cannot meet the
AGPL's terms, and that income helps pay for the project. Offering your code on those terms
needs your permission, and this agreement gives it.
## 1. Definitions
- **"We"** and **"us"** mean Twink Software LLC, a California limited liability company
(Secretary of State entity number B20260081700), under that name or any other, and its
successors and assigns.
- **"You"** means the person who signs this agreement.
- **"Contribution"** means any work of authorship, such as code, documentation, or images,
that you submit to Gearslip, whether before or after you sign. A submission includes a
pull request, patch, or issue attachment sent to a repository or channel we manage. It
excludes anything you mark in writing as "Not a Contribution".
## 2. Copyright license
You grant us, and anyone who receives software from us, a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable license to reproduce, prepare derivative
works of, publicly display, publicly perform, sublicense, and distribute your Contributions
and derivative works of them.
This license lets us offer your Contributions under licenses other than the AGPL, including
commercial licenses.
## 3. Patent license
You grant us, and anyone who receives software from us, a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable patent license to make, have made, use,
offer to sell, sell, import, and otherwise transfer your Contributions. This license covers
only the patent claims you can license that your Contribution, alone or combined with
Gearslip, would infringe.
If anyone sues claiming that Gearslip or a Contribution infringes a patent, the patent
licenses granted to that party for Gearslip end on the date the suit is filed.
## 4. Our promise to you
As long as we distribute a Contribution, we will also offer it under the AGPL or another
license approved by the Open Source Initiative. Your work will always stay available as
free software.
## 5. Your promises
You promise that:
1. **The work is yours.** Each Contribution is your original creation, and you have the
right to grant these licenses.
2. **Your employer agrees.** If your employer has rights in what you create, you have its
permission to contribute on its behalf, or it has waived its rights in your
Contributions.
3. **You name other people's work.** If any part of a Contribution is not your own, you say
so when you submit it, and you name its source and its license.
4. **You'll tell us of changes.** If any of these promises stops being true, you will tell
us.
## 6. No warranty
You provide your Contributions "as is", without warranties or conditions of any kind. You do
not have to provide support for them.
## 7. No obligation
We are not required to use or include any Contribution.
## 8. General
- California law governs this agreement.
- This agreement does not change the rights you have under the AGPL in Gearslip as
released.
- We may publish a new version of this agreement. A new version applies only to
Contributions you make after you sign it; this version keeps covering the ones you made
under it.
## Signing
Most contributors sign on GitHub: the CLA Assistant bot comments on your first pull request,
and you sign there with one click. Your GitHub account then serves as your signature.
To sign by hand instead, fill in the following and send it to us on Matrix. The address is
on the [Seb3thehacker GitHub profile](https://github.com/Seb3thehacker).
Name: ______________________________
GitHub username: ______________________________
Email: ______________________________
Signature: ______________________________
Date: ______________________________
+8 -4
View File
@@ -5,7 +5,7 @@ start there if you aren't sure where to begin.
## Reporting a bug
Open an issue on [GitHub](https://github.com/Seb3thehacker/gearslip/issues) and include:
Open an issue on [GitHub](https://github.com/twinksoftware/gearslip/issues) and include:
- your car's make, model, and year, and whether the head unit is the original one; and
- the log from the drive. Open Logs on the phone app's home screen and tap Share log.
@@ -46,6 +46,10 @@ voice, state things positively, cut needless words, and use the serial comma.
## License
Gearslip is licensed under AGPL-3.0, and your contributions are too. The author also sells a
commercial license, described in [NOTICE.md](NOTICE.md). By opening a pull request, you
agree that your contribution may be offered under that license as well.
Gearslip is licensed under AGPL-3.0, and your contributions are too. Twink Software LLC, which
owns Gearslip, also sells a commercial license, described in [NOTICE.md](NOTICE.md).
Before we can merge your first pull request, you sign the
[Contributor License Agreement](CLA.md). You keep the copyright in your work, and it stays
available as free software; the agreement lets us offer it under the commercial license too.
A bot comments on your pull request with a link, and signing takes one click.
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (C) 2026 seb
Copyright (C) 2026 Twink Software LLC and contributors
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
+1 -1
View File
@@ -6,4 +6,4 @@ You may use, run, and modify Gearslip freely. If you distribute a modified versi
## Commercial licensing
If the AGPL's terms don't work for you, the author sells a separate commercial license. The usual case is a company that wants to use Gearslip privately without publishing its changes. Terms, including any limits on modification, are set case by case. Contact me on Matrix; the address is on my profile.
If the AGPL's terms don't work for you, Twink Software LLC, which owns Gearslip, sells a separate commercial license. The usual case is a company that wants to use Gearslip privately without publishing its changes. Terms, including any limits on modification, are set case by case. Contact us on Matrix; the address is on the [Seb3thehacker GitHub profile](https://github.com/Seb3thehacker).
+91 -28
View File
@@ -4,28 +4,44 @@
An Android Auto alternative built for everyone.
[![forthebadge](https://forthebadge.com/badges/compatibility-club-penguin.svg)](https://forthebadge.com)
[<img src="https://raw.githubusercontent.com/ImranR98/Obtainium/main/assets/graphics/badge_obtainium.png" alt="Get it on Obtainium" height="80">](https://apps.obtainium.imranr.dev/redirect?r=obtainium://add/https://github.com/twinksoftware/gearslip)
Questions, ideas?
[Join the discussion on Matrix](https://matrix.to/#/!mf0_mgBd5KqKpc9FfX_wHp1jFCcwAtPrndIn_oGIyMk?via=matrix.org).
## Which cars work
Most cars work. Starting around 2020, some cars added newer security standards that reject
Gearslip's certificate, and Gearslip can't run in those cars. [CARS.md](CARS.md) lists the
cars drivers have confirmed.
## What Gearslip is
Gearslip is an Android Auto client for the phone. It speaks the Android Auto protocol
directly to a car head unit over USB, with no Google software in the path. Gearslip acts
as a Car App Library host: an app that ships a car interface hands Gearslip a template,
and Gearslip renders that template natively on the car screen. Nothing is mirrored from
the phone. For apps without a car interface, Gearslip offers a separate screen-sharing
mode.
directly to a car head unit, over USB or wirelessly, with no Google software in the path.
Gearslip acts as a Car App Library host: an app that ships a car interface hands Gearslip
a template, and Gearslip renders that template natively on the car screen. Nothing is
mirrored from the phone. For apps without a car interface, Gearslip offers a separate
screen-sharing mode.
| | |
|---|---|
| ![The launcher, with apps grouped into phone, maps, music, and messaging](docs/screenshots/launcher.png) | ![Vela Maps giving turn-by-turn directions, with the music player beside the map](docs/screenshots/navigation.png) |
| Apps sorted by kind. A check marks the ones that work. | Vela Maps navigating, with the player beside it. |
| ![The launcher, with every car app on the phone and the route in the bar below](docs/screenshots/launcher.png) | ![Vela Maps showing the route, with the music player beside the map](docs/screenshots/navigation.png) |
| Your car apps in one place. A red X marks the ones that don't work yet. | Vela Maps navigating, with the player beside it. |
| ![ViVi Music's song, art, and controls in Gearslip's player](docs/screenshots/music.png) | ![A new message card over Organic Maps](docs/screenshots/message.png) |
| ViVi Music in Gearslip's player. | A new message, read aloud at a tap, over the map. |
| ![The dashboard, with the weather, the next calendar event, and recent notifications](docs/screenshots/dashboard.png) | ![DuckDuckGo open in Gearslip's web browser](docs/screenshots/web.png) |
| The weather, your next event, and what you missed. | Browse the web while parked. |
| ![The voice assistant listening over Organic Maps](docs/screenshots/assistant.png) | ![Coffee shops near you in Vela Maps, with the music player beside the map](docs/screenshots/places.png) |
| Ask for a place, a song, or an app. | Find coffee nearby without leaving the map. |
## Features
- **Phone calls.** Dial from a keypad, or search your contacts, then confirm before the
call goes out.
call goes out. When a call comes in, answer or decline it on the car screen.
- **Navigation.** Gearslip hosts real Car App Library map apps on the car screen.
Organic Maps, OsmAnd, and Flitsmeister work today. Vela runs, with rough
edges.
@@ -36,6 +52,9 @@ mode.
doesn't ship a car interface of its own.
- **Weather and the day ahead.** A dashboard screen shows the week's calendar next to
the current and coming weather.
- **Voice.** Tap the mic, or press the talk button on the wheel, and say "navigate to the
nearest gas station", "play Daft Punk", or "call Mom". Say "help" for the full list.
Speech turns into text on the phone, and nothing you say leaves it.
- **Messages.** New messages appear as a small card on the car screen. Gearslip reads
each one aloud, and you reply by voice or with a quick reply. Music pauses while
Gearslip speaks or listens. Speech runs on the phone, so it needs no Google account.
@@ -48,8 +67,9 @@ mode.
## Status
- **The connection is wired only, for now.** Plug the phone into the head unit's USB port;
Gearslip does not yet offer a wireless option.
- **Wired or wireless.** Plug the phone into the car's USB port, or, in a car with wireless
Android Auto, tap Connect wirelessly and pick the car. Pair the phone with the car over
Bluetooth first.
- **Setup walks you through it.** First launch asks for the Android Auto certificate,
then one permission at a time, each with a plain reason and an Allow or Skip. A help
screen inside the car answers anything setup didn't cover.
@@ -61,18 +81,32 @@ mode.
- **Some apps refuse Gearslip.** Spotify, Waze, and Google Messages use a version of
Google's car library that accepts only Android Auto itself. Spotify still plays through
Gearslip's own player.
- **Gearslip works best with cars from before 2020.** Some newer head units reject its
certificate.
- **Calls use the car's Bluetooth,** as they do in Android Auto. Pair the phone with the
car to hear calls through its speakers.
- **Audio reaches the car by capturing playback, not by mirroring the screen.** Android
shows a screen-share consent dialog for this, even though Gearslip never reads the
screen; it is the only way an ordinary app can capture another app's audio. The dialog
appears once per drive.
- **Music reaches the car over Bluetooth or USB.** Setup asks which. Bluetooth is the
default: Android never asks for permission, and your map app's voice reaches the car
too. USB sounds a little clearer, with no delay, but Android shows a screen-share
consent dialog once per drive. Gearslip never reads the screen; capturing playback is
the only way an ordinary app can pass another app's audio along.
- **Gearslip needs no adb, root, or Shizuku.** Every feature relies on an ordinary
Android permission or an on-device consent dialog. The setup guide offers two optional
adb commands: one removes the audio dialog for good, and the other keeps notifications
visible while audio plays.
Android permission or an on-device consent dialog. For USB audio, setup offers two
optional adb commands: one removes the audio dialog for good, and the other keeps
notifications visible while audio plays.
- **Updates come from GitHub.** Once a day, Gearslip checks this repository's latest
release. When a new version is out, it downloads the APK, checks it against GitHub's
SHA-256, and installs it through Android's own installer. Turn off automatic checks in
Settings.
## Verify your download
Check that your copy of Gearslip is genuine with
[AppVerifier](https://github.com/soupslurpr/AppVerifier): share this block to it, or compare it
with the output of `apksigner verify --print-certs`.
```
app.seb3thehacker.gearslip
4C:C5:6E:BD:D1:3A:54:D2:60:EC:A9:B7:E2:B2:AE:6B:30:E2:F2:25:BF:DA:8F:50:8F:07:09:ED:F0:84:94:8B
```
## Usage notes
@@ -81,20 +115,37 @@ turn them on, in setup or in Settings.
- **When they go out.** When you open Gearslip, at most once a day, and after each drive.
Nothing runs in the background.
- **What they carry.** The Gearslip version, plus whatever you tick: the Android version
and whether the phone runs GrapheneOS, the phone model, whether a car connected that
day, and each car's head unit, model, year, screen size, and connection result.
Settings shows the exact note.
- **What they carry.** The Gearslip version and which boxes you ticked, plus whatever
those boxes cover: the Android version and whether the phone runs GrapheneOS, the phone
model, whether a car connected that day, and each car's head unit, model, year, screen
size, and connection result, with whether that drive used the cable or wireless and
whether music went over USB or Bluetooth. If you tick it, how each drive went: how far
the connection got, how long it lasted, and how it ended. Settings shows the exact note. Untick a phone detail, and
the server drops the copy it kept.
- **What they leave out.** Your name, accounts, location, contacts, messages, and
anything you type or play.
- **How phones are counted.** A random number counts each phone once. The server keeps
only a salted hash of it.
- **Where they go.** To a Cloudflare Worker, whose code is in
[stats-worker/](stats-worker/). Cloudflare sees your IP address in delivering a note,
as it would for any website; the Worker doesn't store it.
- **Where they go.** To Gearslip's server. Like any website, it sees your IP address when a
note arrives, and it doesn't store it.
Turn the notes off, and they stop at once.
## How Gearslip is made
Gearslip is my beloved child. I decide what it does and how it looks, and Claude writes nearly
all of the code. I test every change myself before it ships: in a real car, with both a
GrapheneOS phone and a stock Android phone, and on [LIVI](https://github.com/f-io/LIVI), the
open-source head unit software. I watch the phone's temperature too, so a new feature doesn't
cook it. Then I beg my friends to try it in their cars, so it works in as many makes, models,
and years as possible. If something breaks, that's on me, so please open an issue.
## Contributing
Bug reports from real cars help most, and pull requests are welcome.
[CONTRIBUTING.md](CONTRIBUTING.md) explains what to put in a report, which branch to work
on, and the one-time agreement you sign before your first pull request merges.
## Building
```
@@ -113,6 +164,17 @@ keystore. Without them, the debug build still works.
[docs/BUILDING_APPS.md](docs/BUILDING_APPS.md) specifies how to write apps that Gearslip
can render: templated apps, media apps, and the Gearslip app protocol.
## How Gearslip talks to cars
Gearslip is an independent program, written to work with car head units. What it knows of
the protocol comes from public open-source projects such as
[aasdk](https://github.com/opencardev/aasdk) and open-android-auto, and it contains none
of Google's Android Auto code.
Gearslip ships without a certificate. On first launch you choose where yours comes from:
download the one the aasdk project publishes, or import a file of your own. The
certificate stays on your phone. Gearslip never uploads, backs up, or logs it.
## Trademarks
Android, Android Auto, Google Messages, and Waze are trademarks of Google LLC. Spotify is
@@ -121,5 +183,6 @@ any of these companies. All other names belong to their owners.
## License
Gearslip is licensed under AGPL-3.0; see [LICENSE](LICENSE). The author offers a separate
commercial license for private use outside the AGPL's terms; see [NOTICE.md](NOTICE.md).
Gearslip belongs to Twink Software LLC and is licensed under AGPL-3.0; see [LICENSE](LICENSE).
The LLC also sells a separate commercial license for private use outside the AGPL's terms; see
[NOTICE.md](NOTICE.md).
Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

@@ -1,14 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="432" height="432" viewBox="0 0 108 108">
<defs>
<linearGradient id="trail" x1="73.4" y1="46.0" x2="95.8" y2="44.5" gradientUnits="userSpaceOnUse">
<stop offset="0%" stop-color="#8C1F1F" stop-opacity="0"/>
<stop offset="100%" stop-color="#8C1F1F" stop-opacity="0.7"/>
</linearGradient>
</defs>
<rect width="108" height="108" fill="#0c0c0d"/>
<circle cx="54" cy="54" r="52" fill="#131415"/>
<path d="M 73.23,47.15 L 94.77,51.90 L 96.89,37.05 L 73.57,44.78 Z" fill="url(#trail)"/>
<path d="M 54.00,27.00 L 59.99,19.91 Q 61.02,18.69 62.50,19.30 L 72.52,23.45 Q 74.00,24.07 73.87,25.66 L 73.23,33.31 Q 73.09,34.91 73.70,36.39 L 80.39,52.52 Q 81.00,54.00 82.22,55.03 L 88.09,59.99 Q 89.31,61.02 88.70,62.50 L 84.55,72.52 Q 83.93,74.00 82.34,73.87 L 73.09,73.09 Q 73.09,73.09 73.09,73.09 L 73.09,73.09 Q 73.09,73.09 73.09,73.09 L 73.87,82.34 Q 74.00,83.93 72.52,84.55 L 62.50,88.70 Q 61.02,89.31 59.99,88.09 L 54.00,81.00 Q 54.00,81.00 54.00,81.00 L 54.00,81.00 Q 54.00,81.00 54.00,81.00 L 48.01,88.09 Q 46.98,89.31 45.50,88.70 L 35.48,84.55 Q 34.00,83.93 34.13,82.34 L 34.91,73.09 Q 34.91,73.09 34.91,73.09 L 34.91,73.09 Q 34.91,73.09 34.91,73.09 L 25.66,73.87 Q 24.07,74.00 23.45,72.52 L 19.30,62.50 Q 18.69,61.02 19.91,59.99 L 27.00,54.00 Q 27.00,54.00 27.00,54.00 L 27.00,54.00 Q 27.00,54.00 27.00,54.00 L 19.91,48.01 Q 18.69,46.98 19.30,45.50 L 23.45,35.48 Q 24.07,34.00 25.66,34.13 L 34.91,34.91 Q 34.91,34.91 34.91,34.91 L 34.91,34.91 Q 34.91,34.91 34.91,34.91 L 34.13,25.66 Q 34.00,24.07 35.48,23.45 L 45.50,19.30 Q 46.98,18.69 48.01,19.91 L 54.00,27.00 Q 54.00,27.00 54.00,27.00 L 54.00,27.00 Q 54.00,27.00 54.00,27.00 Z" fill="#8B9096"/>
<path d="M 92.78,34.42 L 99.94,37.20 Q 101.43,37.78 101.34,39.37 L 100.67,50.20 Q 100.58,51.80 99.03,52.19 L 91.58,54.08 Q 90.03,54.47 90.13,52.87 L 91.19,35.44 Q 91.29,33.84 92.78,34.42 Z" fill="#8C1F1F"/>
<circle cx="54" cy="54" r="12" fill="#0c0c0d"/>
</svg>

Before

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 10 KiB

-1
View File
@@ -5,7 +5,6 @@
<stop offset="100%" stop-color="#8C1F1F" stop-opacity="0.7"/>
</linearGradient>
</defs>
<rect width="108" height="108" fill="#0c0c0d"/>
<circle cx="54" cy="54" r="52" fill="#131415"/>
<path d="M 73.23,47.15 L 94.77,51.90 L 96.89,37.05 L 73.57,44.78 Z" fill="url(#trail)"/>
<path d="M 54.00,27.00 L 61.02,18.69 L 74.00,24.07 L 73.09,34.91 L 81.00,54.00 L 89.31,61.02 L 83.93,74.00 L 73.09,73.09 L 73.09,73.09 L 74.00,83.93 L 61.02,89.31 L 54.00,81.00 L 54.00,81.00 L 46.98,89.31 L 34.00,83.93 L 34.91,73.09 L 34.91,73.09 L 24.07,74.00 L 18.69,61.02 L 27.00,54.00 L 27.00,54.00 L 18.69,46.98 L 24.07,34.00 L 34.91,34.91 L 34.91,34.91 L 34.00,24.07 L 46.98,18.69 L 54.00,27.00 Z" fill="#8B9096"/>

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 421 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 109 KiB

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 211 KiB

After

Width:  |  Height:  |  Size: 157 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 390 KiB

After

Width:  |  Height:  |  Size: 357 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 161 KiB

After

Width:  |  Height:  |  Size: 165 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 219 KiB

After

Width:  |  Height:  |  Size: 538 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 569 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 86 KiB

After

Width:  |  Height:  |  Size: 87 KiB

+22 -22
View File
@@ -24,6 +24,12 @@ val keystoreProperties = Properties().apply {
).firstOrNull { it.exists() }?.inputStream()?.use { load(it) }
}
// Keeps build output out of the module folder: <root>/build/gearslip/... instead of
// <root>/gearslip/build/..., so everything this project produces lands under one
// root-level build/ directory. Set before android {}, which works out the path to R8's default
// rules from it.
layout.buildDirectory.set(rootProject.layout.buildDirectory.dir("gearslip"))
android {
namespace = "app.seb3thehacker.gearslip"
compileSdk = 37
@@ -32,8 +38,9 @@ android {
applicationId = "app.seb3thehacker.gearslip"
minSdk = 31
targetSdk = 37
versionCode = 8
versionName = "0.2.00-alpha"
versionCode = 10
// -PgearslipVersionName=0.1.99-test makes a build that thinks it's older, to try the updater.
versionName = project.findProperty("gearslipVersionName")?.toString() ?: "0.3.01-alpha"
manifestPlaceholders["appLabel"] = "Gearslip"
// Where opted-in phones send their daily usage note: the stats-worker/ Worker's address,
// set as gearslipStatsUrl in gradle.properties. Left empty, nothing is ever sent.
@@ -56,14 +63,23 @@ android {
buildTypes {
release {
signingConfig = signingConfigs.findByName("release")
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
}
// Its own package, so a dev build installs beside the release one instead of over it.
// Its own package, so a debug build installs beside the release one instead of over it.
// Signed with the release key when it's here: both builds declare the car template
// permission, and Android refuses a second app declaring it under a different key.
debug {
applicationIdSuffix = ".dev"
signingConfigs.findByName("release")?.let { signingConfig = it }
manifestPlaceholders["appLabel"] = "Gearslip Dev"
manifestPlaceholders["appLabel"] = "Gearslip Debug"
// -Pr8 shrinks a debug build like a release one, to test R8 in the debug install.
if (project.hasProperty("r8")) {
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
}
}
}
@@ -71,32 +87,16 @@ android {
buildConfig = true
}
// Keeps build output out of the module folder: <root>/build/gearslip/... instead of
// <root>/gearslip/build/..., so everything this project produces lands under one
// root-level build/ directory.
layout.buildDirectory.set(rootProject.layout.buildDirectory.dir("gearslip"))
compileOptions {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
}
defaultConfig {
// Speech-to-text ships for 64-bit ARM only: every phone Gearslip runs on, and one copy
// of the native library instead of four.
// Speech-to-text (sherpa-onnx, in jniLibs) ships for 64-bit ARM only: every phone
// Gearslip runs on, and one copy of the native library instead of four.
ndk { abiFilters += "arm64-v8a" }
}
ndkVersion = "27.1.12297006"
externalNativeBuild {
cmake {
path = file("src/main/cpp/CMakeLists.txt")
version = "3.22.1"
}
}
androidResources {
// Model weights barely compress, and stored plainly they stream straight out of the APK.
noCompress += "bin"
}
testOptions {
// GearslipLog calls android.util.Log, which is a stub on the local JVM.
+13
View File
@@ -0,0 +1,13 @@
# R8 shrinks the release build; these keep what it can't see being used.
# Names stay readable, so stack traces in shared logs need no mapping file. Shrinking does
# the real work on size.
-dontobfuscate
# Gearslip hosts other apps' car screens: their templates arrive as Bundles, and the car
# library rebuilds them by class name and reflection. Any model class an app might send has to
# survive, used by Gearslip's code or not.
-keep class androidx.car.app.** { *; }
# Speech-to-text: the native library calls back into these classes by name.
-keep class com.k2fsa.sherpa.onnx.** { *; }
+20 -1
View File
@@ -4,6 +4,7 @@
<!-- The projected WebView loads real pages; without this every load fails silently. -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />
<!--
Phone mirroring. Android refuses to hand over a MediaProjection unless a foreground
@@ -19,6 +20,22 @@
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION" />
<!--
Wireless Android Auto: Bluetooth reaches the paired car's Android Auto service, which
hands over its Wi-Fi details; Gearslip then joins that network to reach the car.
-->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.BLUETOOTH" android:maxSdkVersion="30" />
<uses-permission
android:name="android.permission.NEARBY_WIFI_DEVICES"
android:usesPermissionFlags="neverForLocation" />
<!-- Android 17 drops an app's traffic to the car's Wi-Fi without this. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<!-- The dashboard opened from the car clock: today's agenda and current weather. -->
<uses-permission android:name="android.permission.READ_CALENDAR" />
@@ -74,8 +91,10 @@
LAUNCHER query below to find, so the setup guide's Android Auto check needs this to see
whether the package is installed at all. -->
<package android:name="com.google.android.projection.gearhead" />
<!-- Every GrapheneOS install has this; the opt-in usage note uses it to say "GrapheneOS". -->
<!-- The opt-in usage note names the OS: GrapheneOS's setup app and /e/OS's app store give
them away (UsageStats.osName). -->
<package android:name="app.grapheneos.setupwizard" />
<package android:name="foundation.e.apps" />
<intent>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
-36
View File
@@ -1,36 +0,0 @@
# Speech-to-text for voice replies and the assistant: whisper.cpp, pinned to one commit and
# fetched at build time, so nothing outside this folder (cloned-repos included) is needed.
cmake_minimum_required(VERSION 3.22)
project(gearslip_speech CXX C)
set(CMAKE_CXX_STANDARD 17)
# Optimised in debug builds too. Unoptimised, Whisper runs several times slower, and a dev build
# would make voice look far worse than what ships.
foreach(lang C CXX)
set(CMAKE_${lang}_FLAGS_DEBUG "-O3 -DNDEBUG")
endforeach()
include(FetchContent)
FetchContent_Declare(
whisper
URL https://github.com/ggml-org/whisper.cpp/archive/6e4ab854f67f743900934a703d5603419384c961.tar.gz
URL_HASH SHA256=55b585aec5e1bc03e2a5cea61da97c6406fa833996cf7daeed704972273cc039
)
set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_SERVER OFF CACHE BOOL "" FORCE)
set(GGML_OPENMP OFF CACHE BOOL "" FORCE)
set(GGML_NATIVE OFF CACHE BOOL "" FORCE)
# ARMv8.2 with half floats and dot products: every phone that runs Android 12 or later in
# practice, and what makes Whisper fast enough to answer in under a second.
set(GGML_CPU_ARM_ARCH "armv8.2-a+dotprod+fp16" CACHE STRING "" FORCE)
FetchContent_MakeAvailable(whisper)
add_library(gearslip_speech SHARED speech_jni.cpp)
target_compile_options(gearslip_speech PRIVATE -O3)
find_library(LOG_LIB log)
target_link_libraries(gearslip_speech whisper android ${LOG_LIB})
target_link_options(gearslip_speech PRIVATE -Wl,--gc-sections -Wl,-z,max-page-size=16384)
-86
View File
@@ -1,86 +0,0 @@
// The bridge between Kotlin's SpeechEngine and whisper.cpp: load a model, turn one recorded
// phrase into text, free the model. Recording and deciding when a phrase ends stay in Kotlin.
#include <jni.h>
#include <android/asset_manager.h>
#include <android/asset_manager_jni.h>
#include <android/log.h>
#include <algorithm>
#include <cmath>
#include <string>
#include "whisper.h"
#define TAG "GearslipSpeech"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
static whisper_context_params context_params() {
whisper_context_params p = whisper_context_default_params();
p.use_gpu = false;
return p;
}
// Streams a model straight out of the APK, so the bundled one is never copied to storage.
static size_t asset_read(void *ctx, void *output, size_t size) {
int n = AAsset_read(static_cast<AAsset *>(ctx), output, size);
return n > 0 ? static_cast<size_t>(n) : 0;
}
static bool asset_eof(void *ctx) { return AAsset_getRemainingLength64(static_cast<AAsset *>(ctx)) <= 0; }
static void asset_close(void *ctx) { AAsset_close(static_cast<AAsset *>(ctx)); }
extern "C" JNIEXPORT jlong JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeInitAsset(JNIEnv *env, jobject, jobject assets, jstring jpath) {
AAssetManager *manager = AAssetManager_fromJava(env, assets);
const char *path = env->GetStringUTFChars(jpath, nullptr);
AAsset *asset = AAssetManager_open(manager, path, AASSET_MODE_STREAMING);
env->ReleaseStringUTFChars(jpath, path);
if (!asset) return 0;
whisper_model_loader loader = {asset, asset_read, asset_eof, asset_close};
return reinterpret_cast<jlong>(whisper_init_with_params(&loader, context_params()));
}
// Bytes, not a jstring: NewStringUTF wants modified UTF-8 and aborts on some of what Whisper can
// write (four-byte characters), so Kotlin decodes the plain UTF-8 instead.
extern "C" JNIEXPORT jbyteArray JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeTranscribe(
JNIEnv *env, jobject, jlong handle, jfloatArray jsamples, jstring jlang, jint threads) {
auto *ctx = reinterpret_cast<whisper_context *>(handle);
if (!ctx) return env->NewByteArray(0);
jsize n = env->GetArrayLength(jsamples);
jfloat *samples = env->GetFloatArrayElements(jsamples, nullptr);
const char *lang = env->GetStringUTFChars(jlang, nullptr);
whisper_full_params p = whisper_full_default_params(WHISPER_SAMPLING_GREEDY);
p.n_threads = threads;
p.language = lang;
p.translate = false;
p.no_context = true;
p.no_timestamps = true;
p.single_segment = true;
p.print_progress = false;
p.print_realtime = false;
p.print_special = false;
p.print_timestamps = false;
p.suppress_blank = true;
// The ACFT models are trained for a context cut to the audio's own length (1500 frames is
// 30 s, so 50 per second), which is what makes a three-second phrase fast. A little extra
// keeps the last word from being clipped.
p.audio_ctx = std::min(1500, static_cast<int>(std::ceil(n / 320.0)) + 32);
int rc = whisper_full(ctx, p, samples, n);
env->ReleaseStringUTFChars(jlang, lang);
env->ReleaseFloatArrayElements(jsamples, samples, JNI_ABORT);
if (rc != 0) {
LOGI("whisper_full failed: %d", rc);
return env->NewByteArray(0);
}
std::string text;
for (int i = 0; i < whisper_full_n_segments(ctx); i++) text += whisper_full_get_segment_text(ctx, i);
jbyteArray out = env->NewByteArray(static_cast<jsize>(text.size()));
env->SetByteArrayRegion(out, 0, static_cast<jsize>(text.size()), reinterpret_cast<const jbyte *>(text.data()));
return out;
}
extern "C" JNIEXPORT void JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeFree(JNIEnv *, jobject, jlong handle) {
if (handle) whisper_free(reinterpret_cast<whisper_context *>(handle));
}
@@ -9,7 +9,6 @@ object AppSettings {
private const val PREFS = "gearslip_settings"
private const val KEY_STARTUP_URL = "startup_url"
private const val KEY_SEEN_COMPAT_WARNING = "seen_compat_warning"
private const val KEY_SKIPPED_CERT_SETUP = "skipped_cert_setup"
private const val KEY_SEEN_PERMISSIONS_SETUP = "seen_permissions_setup"
private const val KEY_GEARSLIP_ENABLED = "gearslip_enabled"
@@ -32,14 +31,6 @@ object AppSettings {
prefs(context).edit().putString(KEY_STARTUP_URL, value.trim()).apply()
}
/** Whether the newer-car compatibility warning has already been shown and dismissed once. */
fun hasSeenCompatWarning(context: Context): Boolean =
prefs(context).getBoolean(KEY_SEEN_COMPAT_WARNING, false)
fun setSeenCompatWarning(context: Context) {
prefs(context).edit().putBoolean(KEY_SEEN_COMPAT_WARNING, true).apply()
}
/** Whether the certificate setup screen has been deliberately skipped. */
fun hasSkippedCertSetup(context: Context): Boolean =
prefs(context).getBoolean(KEY_SKIPPED_CERT_SETUP, false)
@@ -10,6 +10,26 @@ object Changelog {
class Release(val versionCode: Int, val versionName: String, val changes: List<String>)
val releases = listOf(
Release(
9, "0.3.00-alpha",
listOf(
"Wireless Android Auto: pair the phone with the car once, and leave the cable at home.",
"Voice control is built in. Say \"navigate to a gas station\", \"play\" a song or artist, or open an app by name.",
"\"Navigate to\" picks the nearest place and stops at the map app's start screen, so you only tap Start.",
"Gearslip turns speech into text on the phone, with no Google account. Download the voice models once in setup.",
"The car's rotary knob moves between buttons, so cars without a touchscreen work too.",
"Choose how music reaches the car: Bluetooth, or USB with no delay. Gearslip offers to pair with the car the first time.",
"The car speaks each turn from your map app, and shows the directions on its own display where it can.",
"The car shows what's playing, and music apps get their own like, shuffle and repeat buttons.",
"Pick how the car's buttons look. Change it any time in the car's settings.",
"A new destination mid-route replaces the old route, and switching map apps ends the old app's route.",
"The car warns you when the phone's location is off.",
"Gearslip finds new versions and installs them for you.",
"More cars play music over USB, show a sharper picture, and put taps where you press.",
"On GrapheneOS, Gearslip no longer needs exploit protection compatibility mode.",
"The app is half the size it was.",
),
),
Release(
8, "0.2.00-alpha",
listOf(
@@ -8,13 +8,17 @@ import android.hardware.usb.UsbAccessory
import android.hardware.usb.UsbManager
import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager
import android.net.Uri
import android.os.Bundle
import android.os.ParcelFileDescriptor
import android.provider.Settings
import android.view.MotionEvent
import android.view.Surface
import android.view.WindowManager
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.activity.enableEdgeToEdge
import androidx.activity.result.contract.ActivityResultContracts.StartActivityForResult
import android.content.res.Configuration
@@ -45,6 +49,8 @@ class GearslipActivity : ComponentActivity(), Projection {
private var descriptor: ParcelFileDescriptor? = null
/** The wireless session's socket, when the car came over Wi-Fi instead of USB. */
private var wirelessSocket: java.net.Socket? = null
private var runner: GearslipRunner? = null
private var screenProjector: ScreenProjector? = null
private var worker: Thread? = null
@@ -72,7 +78,13 @@ class GearslipActivity : ComponentActivity(), Projection {
private val audioPermission = registerForActivityResult(
androidx.activity.result.contract.ActivityResultContracts.RequestPermission(),
) { granted ->
if (granted) askAudioCapture() else {
if (granted) askAudioCapture()
else if (!shouldShowRequestPermissionRationale(android.Manifest.permission.RECORD_AUDIO)) {
// Refused twice: Android now says no without showing anything, so the phone must.
GearslipLog.w("audio: record permission blocked - Android no longer asks")
CarAudio.micBlocked()
micBlockedNotice.value = true
} else {
GearslipLog.w("audio: record permission declined")
CarAudio.denied()
}
@@ -85,14 +97,22 @@ class GearslipActivity : ComponentActivity(), Projection {
} else {
GearslipLog.w("audio: capture consent declined")
CarAudio.denied()
if (!CarSettings.audioChoiceAsked) offerBluetooth.value = true
}
}
/** Raised when the microphone permission is blocked; see [MicBlockedDialog]. */
private val micBlockedNotice = kotlinx.coroutines.flow.MutableStateFlow(false)
/** Raised once, after the first "no" to the capture dialog; see [AudioChoiceDialog]. */
private val offerBluetooth = kotlinx.coroutines.flow.MutableStateFlow(false)
private val detachReceiver = object : BroadcastReceiver() {
override fun onReceive(context: Context?, intent: Intent?) {
if (intent?.action == UsbManager.ACTION_USB_ACCESSORY_DETACHED) {
GearslipLog.w("accessory detached")
closeAccessory()
// A wireless session has no accessory to lose; leave it running.
if (wirelessSocket == null) closeAccessory()
}
}
}
@@ -115,6 +135,9 @@ class GearslipActivity : ComponentActivity(), Projection {
GearslipLog.init(this)
GearslipLog.installCrashHandler()
GearslipLog.i("Gearslip ready")
app.seb3thehacker.gearslip.wireless.WirelessLink.onConnected = { socket ->
runOnUiThread { startWireless(socket) }
}
CarSettings.init(this)
AppSettings.applyGearslipEnabled(this)
// A capture that died without tidying up would leave notifications exposed to every
@@ -127,7 +150,33 @@ class GearslipActivity : ComponentActivity(), Projection {
setContent {
GearslipTheme {
GearslipApp(onDisconnect = ::closeAccessory, onRequestCallScreening = ::requestCallScreeningRole)
GearslipApp(onDisconnect = { closeAccessory() }, onRequestCallScreening = ::requestCallScreeningRole)
val offer by offerBluetooth.collectAsState()
if (offer) {
fun close() {
CarSettings.markAudioChoiceAsked()
offerBluetooth.value = false
}
app.seb3thehacker.gearslip.ui.AudioChoiceDialog(
onUseBluetooth = {
CarSettings.setPipeAudio(false)
close()
},
onKeepAsking = ::close,
)
}
val micNotice by micBlockedNotice.collectAsState()
if (micNotice) {
app.seb3thehacker.gearslip.ui.MicBlockedDialog(
onOpenSettings = {
micBlockedNotice.value = false
startActivity(
Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS, Uri.fromParts("package", packageName, null)),
)
},
onDismiss = { micBlockedNotice.value = false },
)
}
}
}
@@ -152,6 +201,11 @@ class GearslipActivity : ComponentActivity(), Projection {
super.onStart()
Prefetch.warm(this)
UsageStats.onAppOpened(this)
app.seb3thehacker.gearslip.update.UpdateChecker.onAppOpened(this)
// Back from the app's settings with the microphone turned on: try car audio again.
if (checkSelfPermission(android.Manifest.permission.RECORD_AUDIO) == android.content.pm.PackageManager.PERMISSION_GRANTED) {
CarAudio.micAllowed()
}
}
override fun onNewIntent(intent: Intent) {
@@ -161,6 +215,7 @@ class GearslipActivity : ComponentActivity(), Projection {
override fun onDestroy() {
super.onDestroy()
app.seb3thehacker.gearslip.wireless.WirelessLink.onConnected = null
runCatching { unregisterReceiver(detachReceiver) }
PhoneMirror.onStartRequested = null
PhoneMirror.onTokenReady = null
@@ -296,16 +351,44 @@ class GearslipActivity : ComponentActivity(), Projection {
worker = Thread { runner.run() }.apply { name = "gearslip"; start() }
}
/** A wireless session: the same runner, on the socket [WirelessLink] opened to the car. */
private fun startWireless(socket: java.net.Socket) {
closeAccessory(stopWireless = false)
if (!AppSettings.gearslipEnabled(this)) {
GearslipLog.i("wireless: Gearslip is switched off - not connecting")
runCatching { socket.close() }
return
}
GearslipLog.i("wireless: starting a session over Wi-Fi to ${socket.inetAddress?.hostAddress}:${socket.port}")
wirelessSocket = socket
val runner = GearslipRunner(
input = socket.getInputStream(),
output = socket.getOutputStream(),
identityProvider = { CertProvider.load(this) },
projection = this,
vehicleProfileFor = { info -> VehicleProfiles.find(VehicleProfiles.load(this), info) },
wireless = true,
)
this.runner = runner
worker = Thread { runner.run() }.apply { name = "gearslip"; start() }
}
private fun describe(accessory: UsbAccessory) {
GearslipLog.i("accessory: manufacturer=${accessory.manufacturer} model=${accessory.model}")
GearslipLog.i(" version=${accessory.version} description=${accessory.description}")
}
private fun closeAccessory() {
private fun closeAccessory() = closeAccessory(stopWireless = true)
/** Ends the session, wired or wireless. [stopWireless] also drops the car's Bluetooth and Wi-Fi. */
private fun closeAccessory(stopWireless: Boolean) {
runner?.stop()
SessionReport.print()
runCatching { descriptor?.close() }
descriptor = null
runCatching { wirelessSocket?.close() }
wirelessSocket = null
if (stopWireless) app.seb3thehacker.gearslip.wireless.WirelessLink.stop(this)
runner = null
}
@@ -337,6 +420,8 @@ class GearslipActivity : ComponentActivity(), Projection {
}
override fun onProjectionStopped() {
// Before the car UI goes, while the media connection is still up.
app.seb3thehacker.gearslip.car.CarServices.pausePhoneMedia()
runOnUiThread {
PhoneMirror.requestStop()
app.seb3thehacker.gearslip.car.CarKeys.focusSink = null
@@ -3,6 +3,12 @@ package app.seb3thehacker.gearslip
import android.os.Build
import app.seb3thehacker.gearslip.audio.AudioLink
import app.seb3thehacker.gearslip.audio.CarMic
import app.seb3thehacker.gearslip.media.CarMediaStatus
import app.seb3thehacker.gearslip.car.BluetoothAudioCheck
import app.seb3thehacker.gearslip.car.LocationCheck
import app.seb3thehacker.gearslip.car.CarBluetooth
import app.seb3thehacker.gearslip.car.ClusterNav
import app.seb3thehacker.gearslip.car.DirectionsVoice
import app.seb3thehacker.gearslip.car.CarAssistant
import app.seb3thehacker.gearslip.car.CarEnvironment
import app.seb3thehacker.gearslip.car.CarKeys
@@ -33,6 +39,8 @@ class GearslipRunner(
private val identityProvider: () -> CertProvider.Identity,
private val projection: Projection? = null,
private val vehicleProfileFor: (ServiceDiscovery.HeadUnitInfo) -> VehicleProfile? = { null },
/** Over Wi-Fi rather than USB: the video is sent smaller so the link keeps up. */
private val wireless: Boolean = false,
) {
private val log = GearslipLog.tagged("PROTO")
@@ -43,10 +51,15 @@ class GearslipRunner(
private lateinit var identityTask: FutureTask<PhoneTls>
/** Sends every message, in order, so no other thread ever blocks on the USB link. See [send]. */
private val writer = java.util.concurrent.Executors.newSingleThreadExecutor { Thread(it, "gearslip-writer") }
private val writer = java.util.concurrent.Executors.newSingleThreadExecutor {
// Sound goes out through here too, so it gets audio's priority: late writes are skips.
Thread({ android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_AUDIO); it.run() }, "gearslip-writer")
}
/** Waits for the certificate only when TLS is first needed; see [run]. */
private val tls: PhoneTls by lazy { identityTask.get() }
private var authenticationFailed = false
@Volatile private var stopped = false
@Volatile private var state = State.WAIT_VERSION
@Volatile private var lastProgress = System.currentTimeMillis()
@@ -68,8 +81,12 @@ class GearslipRunner(
@Volatile private var inputChannelId: Int = -1
@Volatile private var inputKeycodes: List<Int> = emptyList()
@Volatile private var sensorChannelId: Int = -1
@Volatile private var mediaStatus: CarMediaStatus? = null
@Volatile private var clusterNav: ClusterNav? = null
private var sensorTypes: List<Int> = emptyList()
@Volatile private var audioLink: AudioLink? = null
@Volatile private var guidanceLink: AudioLink? = null
@Volatile private var vendorChannelId = -1
private var audioMessagesSeen = 0
private var touchWidth = 0
private var touchHeight = 0
@@ -82,7 +99,9 @@ class GearslipRunner(
@Volatile private var lastKeyframeSentAt = 0L
fun run() {
SessionReport.begin()
// This thread reads the car's acks; audio waits on them, so it can't be left at the back.
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_AUDIO)
SessionReport.begin(wireless)
log.i("--- session starting ---")
SessionStatus.connecting("Connected to the head unit")
log.i("device=${Build.MANUFACTURER} ${Build.MODEL} android=${Build.VERSION.RELEASE} (sdk ${Build.VERSION.SDK_INT})")
@@ -107,6 +126,7 @@ class GearslipRunner(
try {
while (running) {
val read = input.read(chunk)
if (stopped) return
if (read < 0) {
log.w("head unit closed the connection (EOF) while in state $state")
reportDisconnect()
@@ -120,16 +140,19 @@ class GearslipRunner(
}
}
} catch (t: Throwable) {
if (stopped) return
log.e("transport failed in state $state", t)
SessionReport.fail(SessionReport.Category.USB, "transport failed", t, state.name)
reportDisconnect()
} finally {
running = false
writer.shutdownNow()
log.flush()
}
}
fun stop() {
stopped = true
running = false
writer.shutdownNow()
CarFocus.request = null
@@ -139,8 +162,17 @@ class GearslipRunner(
videoSource = null
audioLink?.close()
audioLink = null
guidanceLink?.close()
guidanceLink = null
CarSensors.clear()
CarMic.current = null
CarBluetooth.onCarDisconnected()
BluetoothAudioCheck.onCarDisconnected()
LocationCheck.onCarDisconnected()
mediaStatus?.stop()
mediaStatus = null
clusterNav?.stop()
clusterNav = null
lastSensorNight = null
CarEnvironment.setSensorNight(null) // back to the clock until a car says otherwise
projection?.onProjectionStopped()
@@ -148,6 +180,8 @@ class GearslipRunner(
}
private fun handleFrame(frame: Frames.Frame) {
// A rejected handshake cannot consume later messages as if authentication succeeded.
if (authenticationFailed || stopped) return
val payload = if (frame.encrypted) {
if (!tls.handshakeComplete) {
log.w("encrypted frame arrived before the handshake finished - ignoring")
@@ -176,6 +210,16 @@ class GearslipRunner(
onInputMessage(messageId, body)
} else if (frame.channel == sensorChannelId) {
onSensorMessage(messageId, body)
} else if (frame.channel == clusterNav?.channelId) {
clusterNav?.onMessage(messageId, body)
} else if (frame.channel == mediaStatus?.channelId) {
mediaStatus?.onMessage(messageId, body)
} else if (frame.channel == guidanceLink?.channelId) {
guidanceLink?.onMessage(messageId, body)
} else if (frame.channel == CarBluetooth.channelId) {
CarBluetooth.onMessage(messageId, body)
} else if (frame.channel == vendorChannelId) {
onVendorMessage(messageId, body)
} else if (frame.channel == CarMic.current?.channelId) {
CarMic.current?.onMessage(messageId, body)
} else if (frame.channel == audioLink?.channelId) {
@@ -225,7 +269,7 @@ class GearslipRunner(
}
log.w("<- ByeByeRequest: reason=$reason ($name) in state $state")
log.hex(" byebye raw", body)
SessionReport.fail(SessionReport.Category.BYEBYE, "ByeByeRequest reason=$reason ($name)", state.name)
SessionReport.fail(SessionReport.Category.BYEBYE, "ByeByeRequest reason=$reason ($name)", state.name, "bye $reason")
send(MSG_BYEBYE_RESPONSE, ByteArray(0), encrypted = true)
log.i("-> ByeByeResponse")
}
@@ -240,11 +284,10 @@ class GearslipRunner(
log.i("<- VersionRequest: head unit speaks $major.$minor")
SessionReport.protocolVersion(major, minor)
// PROTOCOL-DOWNGRADE EXPERIMENT. The 2018 Uconnect speaks 1.3 and ACCEPTS the JVC cert;
// the 2025 unit speaks 4.3 and rejects it at AuthComplete (-3). If the 4.3 identity check
// is tied to the *negotiated* protocol version rather than baked into the firmware,
// claiming an older version makes the newer unit skip the check entirely. CLAIM_VERSION
// = null restores the previous echo-the-head-unit behaviour.
// Always 1.3, on purpose. Every car that accepts Gearslip was tested this way, 1.5 units
// included. Echoing the car's own version was tried too: a 4.3 unit rejected the
// certificate either way, so the version doesn't decide acceptance. The log says so, since
// a car offering 1.5 and getting 1.3 back looks like a bug to anyone reading it.
val claim = CLAIM_VERSION ?: (major to minor)
val response = byteArrayOf(
((claim.first shr 8) and 0xFF).toByte(), (claim.first and 0xFF).toByte(),
@@ -254,7 +297,10 @@ class GearslipRunner(
send(MSG_VERSION_RESPONSE, response, encrypted = false)
log.i(
"-> VersionResponse: ${claim.first}.${claim.second} status=0 (STATUS_SUCCESS)" +
if (CLAIM_VERSION != null) " [FORCED; head unit offered $major.$minor]" else "",
if (CLAIM_VERSION != null && claim != (major to minor)) {
" - intended: Gearslip always answers ${claim.first}.${claim.second}, which cars " +
"offering $major.$minor accept too"
} else "",
)
state = State.TLS_HANDSHAKE
SessionStatus.connecting("Securing the link")
@@ -262,10 +308,14 @@ class GearslipRunner(
}
private fun onHandshake(body: ByteArray) {
if (state != State.TLS_HANDSHAKE && state != State.WAIT_AUTH) return
log.i("<- EncapsulatedSSL (${body.size} bytes)")
val replies = try {
tls.pumpHandshake(body)
val preparedTls = tls
if (stopped) return
preparedTls.pumpHandshake(body)
} catch (t: Throwable) {
if (stopped) return
log.e("TLS handshake threw", t)
SessionReport.fail(SessionReport.Category.TLS, "handshake threw", t, state.name)
log.verdict(
@@ -278,9 +328,11 @@ class GearslipRunner(
"The head unit aborted the secure handshake. It validates the phone's certificate.",
)
state = State.DONE
authenticationFailed = true
SessionReport.print()
return
}
if (stopped) return
replies.forEach {
send(MSG_ENCAPSULATED_SSL, it, encrypted = false)
log.i("-> EncapsulatedSSL (${it.size} bytes)")
@@ -293,6 +345,7 @@ class GearslipRunner(
private fun onAuthComplete(body: ByteArray) {
val status = Protobuf.readInt32Field(body, 1)
if (state != State.WAIT_AUTH && !(state == State.TLS_HANDSHAKE && status in listOf(-2, -3))) return
log.i("<- AuthComplete: status=$status")
status?.let { SessionReport.auth(it) }
@@ -304,7 +357,7 @@ class GearslipRunner(
sendServiceDiscoveryRequest()
}
-2 -> {
SessionReport.fail(SessionReport.Category.CERTIFICATE, "AuthComplete status -2 (certificate error)", state.name)
SessionReport.fail(SessionReport.Category.CERTIFICATE, "AuthComplete status -2 (certificate error)", state.name, "auth -2")
log.verdict(
"NOT VIABLE (STATUS_CERTIFICATE_ERROR)",
"The head unit explicitly rejected the phone's self-signed certificate. " +
@@ -316,9 +369,10 @@ class GearslipRunner(
"The head unit refused the phone's certificate (certificate error).",
)
state = State.DONE
authenticationFailed = true
}
-3 -> {
SessionReport.fail(SessionReport.Category.AUTH, "AuthComplete status -3 (authentication failure)", state.name)
SessionReport.fail(SessionReport.Category.AUTH, "AuthComplete status -3 (authentication failure)", state.name, "auth -3")
log.verdict(
"NOT VIABLE (STATUS_AUTHENTICATION_FAILURE)",
"Authentication rejected. Broader than a pure certificate error, but the " +
@@ -329,9 +383,10 @@ class GearslipRunner(
"Authentication failed. The head unit does not trust the certificate in use.",
)
state = State.DONE
authenticationFailed = true
}
else -> {
SessionReport.fail(SessionReport.Category.AUTH, "AuthComplete status $status (unexpected)", state.name)
SessionReport.fail(SessionReport.Category.AUTH, "AuthComplete status $status (unexpected)", state.name, "auth $status")
log.verdict(
"INCONCLUSIVE (AuthComplete status=$status)",
"Unexpected status. Check MessageStatus.proto in aasdk for the meaning " +
@@ -339,6 +394,7 @@ class GearslipRunner(
)
SessionStatus.failed("Unexpected response", "The head unit answered authentication with status $status.")
state = State.DONE
authenticationFailed = true
}
}
}
@@ -399,7 +455,7 @@ class GearslipRunner(
val video = ServiceDiscovery.findVideoService(serviceDiscoveryResponse)
if (video == null) {
log.w("no video service in the discovery response - cannot start Phase A")
SessionReport.fail(SessionReport.Category.VIDEO, "head unit advertised no video service", state.name)
SessionReport.fail(SessionReport.Category.VIDEO, "head unit advertised no video service", state.name, "no video service")
return
}
videoChannelId = video.serviceId
@@ -415,8 +471,13 @@ class GearslipRunner(
openChannel(video.serviceId)
startAudioChannel(serviceDiscoveryResponse)
startGuidanceChannel(serviceDiscoveryResponse)
startSensorChannel(serviceDiscoveryResponse)
startMicChannel(serviceDiscoveryResponse)
startMediaStatusChannel(serviceDiscoveryResponse)
startClusterNavChannel(serviceDiscoveryResponse)
startBluetooth(serviceDiscoveryResponse)
startVendorChannel(serviceDiscoveryResponse)
val input = ServiceDiscovery.findInputService(serviceDiscoveryResponse, video.displayId)
if (input == null) {
@@ -434,7 +495,7 @@ class GearslipRunner(
}
}
/** Media audio rides its own sink; guidance, system and telephony sinks are left closed. */
/** Media audio rides its own sink; spoken directions take the guidance sink, and system and telephony stay closed. */
private fun startAudioChannel(serviceDiscoveryResponse: ByteArray) {
val sinks = ServiceDiscovery.findAudioServices(serviceDiscoveryResponse)
sinks.forEach { sink ->
@@ -451,7 +512,7 @@ class GearslipRunner(
return
}
if (media.codecType != ServiceDiscovery.CODEC_PCM) {
log.w("the car offers no PCM media sink, only ${media.codecName}; it may play our PCM as silence")
log.w("the car offers no PCM media sink, only ${media.codecName}; music goes as AAC")
}
val first = media.configs.firstOrNull()
SessionReport.audio(
@@ -462,10 +523,34 @@ class GearslipRunner(
media,
sendOnChannel = { id, body -> send(id, body, encrypted = true, channel = media.serviceId) },
sendControl = { id, body -> send(id, body, encrypted = true) },
// The driver picked the radio or another source on the car: pause the phone's music
// as Android Auto does, instead of letting it play on to no one.
onTakenBack = { CarServices.pausePhoneMedia("the car switched to another source") },
)
openChannel(media.serviceId)
}
/** The car's directions channel, where [DirectionsVoice] reads out each turn. */
private fun startGuidanceChannel(serviceDiscoveryResponse: ByteArray) {
val guidance = ServiceDiscovery.findAudioServices(serviceDiscoveryResponse)
.filter { it.streamType == ServiceDiscovery.STREAM_GUIDANCE }
.firstOrNull { it.codecType == ServiceDiscovery.CODEC_PCM } ?: run {
log.i("no PCM directions sink - turns won't be spoken")
return
}
guidanceLink = AudioLink(
guidance,
sendOnChannel = { id, body -> send(id, body, encrypted = true, channel = guidance.serviceId) },
sendControl = { id, body -> send(id, body, encrypted = true) },
tag = "guidance",
sessionId = GUIDANCE_SESSION_ID,
asksFocus = false,
onReady = { DirectionsVoice.attach(it) },
onClose = { DirectionsVoice.detach(it) },
)
openChannel(guidance.serviceId)
}
/** The car's microphone, for voice: opened now, recording only while something listens. */
private fun startMicChannel(serviceDiscoveryResponse: ByteArray) {
val mic = ServiceDiscovery.findMicService(serviceDiscoveryResponse) ?: run {
@@ -474,10 +559,75 @@ class GearslipRunner(
}
val config = mic.configs.first()
log.i("car microphone: channel=${mic.serviceId} ${mic.codecName} ${config.sampleRate}Hz/${config.bits}bit/x${config.channels}")
CarMic.current = CarMic(mic) { id, body -> send(id, body, encrypted = true, channel = mic.serviceId) }
CarMic.current = CarMic(
mic,
sendOnChannel = { id, body -> send(id, body, encrypted = true, channel = mic.serviceId) },
// VoiceSessionNotification { status = 1 START, 2 END }, on the control channel.
voiceSession = { start ->
send(MSG_VOICE_SESSION_NOTIFICATION, Protobuf.varintField(1, if (start) 1L else 2L), encrypted = true)
},
)
openChannel(mic.serviceId)
}
/** What's playing, for the car's own screens: the cluster, the media source page. */
private fun startMediaStatusChannel(serviceDiscoveryResponse: ByteArray) {
val channel = ServiceDiscovery.findMediaStatusChannel(serviceDiscoveryResponse) ?: run {
log.i("no media status channel - the car's own screens won't show what's playing")
return
}
log.i("media status channel: $channel")
mediaStatus = CarMediaStatus(channel) { id, body -> send(id, body, encrypted = true, channel = channel) }
openChannel(channel)
}
/** Bluetooth pairing, for calls: the channel only opens if the driver asks to pair. */
/**
* GM's own channel ("com.gm.vip"), opened in GM cars only. Issue #18: a 2021 Bolt ends every
* session about 25 s in, and of the channels it offers, this is the one a Dacia that stays
* connected doesn't have. What GM sends on it is private, so Gearslip only opens it and logs
* what arrives, by message id and size.
*/
private fun startVendorChannel(serviceDiscoveryResponse: ByteArray) {
val vendors = ServiceDiscovery.findVendorServices(serviceDiscoveryResponse)
vendors.forEach { log.i("vendor channel ${it.serviceId}: ${it.name}") }
val gm = vendors.firstOrNull { it.name.startsWith("com.gm.") } ?: return
vendorChannelId = gm.serviceId
log.i("-> ChannelOpenRequest on GM's channel ${gm.serviceId}")
openChannel(gm.serviceId)
}
private fun onVendorMessage(messageId: Int, body: ByteArray) {
if (messageId == MSG_CHANNEL_OPEN_RESPONSE) {
log.i("<- GM channel open: status=${Protobuf.readInt32Field(body, 1) ?: 0}")
} else {
log.i("<- GM channel message id=$messageId (${body.size} bytes)")
}
}
private fun startBluetooth(serviceDiscoveryResponse: ByteArray) {
val bluetooth = ServiceDiscovery.findBluetoothService(serviceDiscoveryResponse) ?: run {
log.i("no Bluetooth service - the car can't be paired from here")
return
}
CarBluetooth.onCarConnected(
bluetooth,
open = { openChannel(bluetooth.serviceId) },
send = { id, body -> send(id, body, encrypted = true, channel = bluetooth.serviceId) },
)
}
/** Turn-by-turn for the instrument cluster: the next turn, its road, and how far. */
private fun startClusterNavChannel(serviceDiscoveryResponse: ByteArray) {
val channel = ServiceDiscovery.findNavStatusChannel(serviceDiscoveryResponse) ?: run {
log.i("no navigation status channel - the cluster won't show turns")
return
}
log.i("navigation status channel: $channel")
clusterNav = ClusterNav(channel) { id, body -> send(id, body, encrypted = true, channel = channel) }
openChannel(channel)
}
/**
* Every sensor type the head unit advertised, subscribed to at once - there is no screen
* to pick and choose yet, this is meant to show everything reachable. [CarSensors] answers
@@ -599,8 +749,8 @@ class GearslipRunner(
* right-hand third of the screen away entirely - taps on a right-edge button landed on
* the middle one. The head unit maps its own panel onto our video before reporting.
*
* So: inject as-is, clamped to the video bounds. The advertised touchscreen size is kept
* only for diagnostics.
* So: inject them unscaled, shifted past the cropped margin (see below) and clamped to the
* video. The advertised touchscreen size is kept only for diagnostics.
*/
private fun onInputMessage(messageId: Int, body: ByteArray) {
when (messageId) {
@@ -631,13 +781,20 @@ class GearslipRunner(
val action = Wire.varint(touchFields, 3)?.toInt() ?: 0
val pointers = Wire.allBytes(touchFields, 1)
if (pointers.isEmpty()) return
val size = videoConfigs.getOrNull(selectedConfigIndex ?: 0)?.pixelSize()
val config = videoConfigs.getOrNull(selectedConfigIndex ?: 0)
val size = config?.pixelSize()
// The car crops its margin off the frame and measures taps from the corner of what
// it shows, so they come in short by half the margin: the 2024 VW (margin 240) sent
// y=6, which is inside the cropped band, and the Uconnect's never passed 351 on a
// 480-high frame with a 130 margin. Without this every tap landed 120 px too high
// and the VW couldn't press past its first screen.
val shift = config?.marginInsets() ?: Insets.NONE
val points = pointers.mapIndexedNotNull { index, pointer ->
val fields = Wire.fields(pointer)
val rawX = Wire.varint(fields, 1)?.toInt() ?: return@mapIndexedNotNull null
val rawY = Wire.varint(fields, 2)?.toInt() ?: return@mapIndexedNotNull null
val id = Wire.varint(fields, 3)?.toInt() ?: index
TouchPoint(id, clamp(rawX, size?.first), clamp(rawY, size?.second))
TouchPoint(id, clamp(rawX + shift.left, size?.first), clamp(rawY + shift.top, size?.second))
}
if (points.isEmpty()) return
val actionIndex = (Wire.varint(touchFields, 2)?.toInt() ?: 0).coerceIn(0, points.size - 1)
@@ -795,13 +952,17 @@ class GearslipRunner(
/**
* The offered config that best fills the car's screen: the smallest whose visible width
* reaches most of the touchscreen's, else the largest there is. Always taking the smallest
* put 800x480 on a 1540x720 Audi screen and stretched it. With no touchscreen size to go
* by, the smallest is still the safe choice.
* put 800x480 on a 1540x720 Audi screen and stretched it. A car that gives no touchscreen
* size gets the largest up to 1080p: a car only offers sizes it can show, and the 2024 VW
* (touchscreen 0x0) blurred 800x480 across a 1080p screen.
*/
private fun fitToScreen(offered: List<Int>): Int? {
val known = offered.filter { videoConfigs.getOrNull(it)?.pixelSize() != null }
fun pixels(idx: Int) = videoConfigs[idx].pixelCount()
if (touchWidth <= 0) return known.minByOrNull(::pixels)
if (touchWidth <= 0) {
return known.filter { pixels(it) <= 1920 * 1080 }.maxByOrNull(::pixels)
?: known.minByOrNull(::pixels)
}
val wide = known.filter { (videoConfigs[it].visibleSize()?.first ?: 0) >= touchWidth * 8 / 10 }
return wide.minByOrNull(::pixels) ?: known.maxByOrNull(::pixels)
}
@@ -836,8 +997,10 @@ class GearslipRunner(
videoSource = VideoSource(
width = width,
height = height,
frameRate = if (config.frameRate == 1) 60 else 30,
// 60fps only over a cable: over Wi-Fi it doubles the data for little the driver sees.
frameRate = if (config.frameRate == 1 && !wireless) 60 else 30,
mode = if (useProjection) VideoSource.Mode.SURFACE else VideoSource.Mode.TEST_CARD,
wireless = wireless,
onCodecConfig = { csd ->
lastCodecConfig = csd
send(MSG_MEDIA_CODEC_CONFIG, csd, encrypted = true, channel = videoChannelId)
@@ -854,6 +1017,8 @@ class GearslipRunner(
if (projection != null && surface != null) {
projection.onSurfaceReady(surface, width, height, config.density)
SessionStatus.projecting()
BluetoothAudioCheck.onProjecting()
LocationCheck.onProjecting()
}
}.onFailure { e -> log.e("encoder start failed", e) }
}
@@ -1065,10 +1230,10 @@ class GearslipRunner(
private fun reportDisconnect() {
when (state) {
State.WAIT_VERSION -> SessionReport.fail(SessionReport.Category.NO_HANDSHAKE, "no VersionRequest arrived", state.name)
State.TLS_HANDSHAKE -> SessionReport.fail(SessionReport.Category.TLS, "head unit dropped the link during TLS", state.name)
State.WAIT_AUTH -> SessionReport.fail(SessionReport.Category.AUTH, "dropped after TLS, before AuthComplete", state.name)
State.WAIT_SDR -> SessionReport.fail(SessionReport.Category.SERVICE_DISCOVERY, "accepted, then dropped before service discovery", state.name)
State.WAIT_VERSION -> SessionReport.fail(SessionReport.Category.NO_HANDSHAKE, "no VersionRequest arrived", state.name, "closed")
State.TLS_HANDSHAKE -> SessionReport.fail(SessionReport.Category.TLS, "head unit dropped the link during TLS", state.name, "closed")
State.WAIT_AUTH -> SessionReport.fail(SessionReport.Category.AUTH, "dropped after TLS, before AuthComplete", state.name, "closed")
State.WAIT_SDR -> SessionReport.fail(SessionReport.Category.SERVICE_DISCOVERY, "accepted, then dropped before service discovery", state.name, "closed")
State.DONE -> {}
}
when (state) {
@@ -1155,9 +1320,9 @@ class GearslipRunner(
private companion object {
/**
* Force the protocol version Gearslip claims in VersionResponse, as (major, minor), or
* null to echo the head unit. 1.3 is what the 2018 Uconnect (which accepts the JVC cert)
* speaks. This is the 2025 identity-check downgrade experiment: see onVersionRequest.
* The protocol version Gearslip answers every car with, as (major, minor), or null to echo
* the head unit's. Kept at 1.3 because every working car was tested with it; see
* onVersionRequest.
*/
val CLAIM_VERSION: Pair<Int, Int>? = 1 to 3
@@ -1172,6 +1337,7 @@ class GearslipRunner(
const val MSG_PING_RESPONSE = 12
const val MSG_BYEBYE_REQUEST = 15
const val MSG_BYEBYE_RESPONSE = 16
const val MSG_VOICE_SESSION_NOTIFICATION = 17
const val MSG_CHANNEL_OPEN_REQUEST = 7
const val MSG_CHANNEL_OPEN_RESPONSE = 8
@@ -1212,6 +1378,7 @@ class GearslipRunner(
const val ACK_STALL_MS = 2_000L
const val AUDIO_TRACE_MESSAGES = 6
const val GUIDANCE_SESSION_ID = 1
const val PERIODIC_KEYFRAME_MS = 5_000L
}
}
@@ -31,6 +31,9 @@ object Protobuf {
return varint(((field shl 3) or 2).toLong()) + varint(bytes.size.toLong()) + bytes
}
fun bytesField(field: Int, bytes: ByteArray): ByteArray =
varint(((field shl 3) or 2).toLong()) + varint(bytes.size.toLong()) + bytes
fun varintField(field: Int, value: Long): ByteArray =
varint((field shl 3).toLong()) + varint(value)
@@ -198,6 +198,12 @@ class ScreenProjector(private val context: Context) {
*/
fun dispatchKey(keyCode: Int, down: Boolean) {
val target = root ?: return
// Keys handed straight to the view skip the step where Android leaves touch mode, and in
// touch mode Compose's buttons refuse focus, so the knob had nothing to move between.
if (target.isInTouchMode) {
target.requestFocusFromTouch()
GearslipLog.i("keys: left touch mode for the knob/D-pad (now in touch mode: ${target.isInTouchMode})")
}
// Nothing on the display takes keys until something in it holds focus.
if (target.findFocus() == null) target.requestFocus()
val shift = keyCode == app.seb3thehacker.gearslip.car.CarKeys.KEYCODE_SHIFT_TAB
@@ -110,6 +110,21 @@ object ServiceDiscovery {
return media.firstOrNull { it.codecType == CODEC_PCM } ?: media.firstOrNull()
}
/**
* The channel the car takes "now playing" on (MediaPlaybackStatusService, Service field 9),
* which feeds its cluster and media screens. It carries no settings. Null when the car has none.
*/
fun findMediaStatusChannel(response: ByteArray): Int? = findChannelWith(response, 9)
/** The instrument cluster's turn-by-turn channel (NavigationStatusService, Service field 8). */
fun findNavStatusChannel(response: ByteArray): Int? = findChannelWith(response, 8)
private fun findChannelWith(response: ByteArray, serviceField: Int): Int? =
Wire.allBytes(Wire.fields(response), 1).firstNotNullOfOrNull { channel ->
val service = Wire.fields(channel)
if (service.any { it.number == serviceField }) Wire.varint(service, 1)?.toInt() else null
}
/**
* The car's microphone, from MediaSourceService (Service field 5):
* MediaSourceService { MediaCodecType available_type = 1; AudioConfiguration audio_config = 2;
@@ -138,8 +153,45 @@ object ServiceDiscovery {
return null
}
/**
* The car's Bluetooth, from BluetoothService (Service field 6):
* BluetoothService { string car_address = 1; repeated BluetoothPairingMethod methods = 2 [packed]; }
* Null when the car offers no pairing over the link.
*/
class BluetoothService(val serviceId: Int, val carAddress: String, val pairingMethods: List<Int>)
/** A carmaker's own channel (service field 12), for its apps on the phone: GM's is "com.gm.vip". */
class VendorService(val serviceId: Int, val name: String)
fun findVendorServices(response: ByteArray): List<VendorService> =
Wire.allBytes(Wire.fields(response), 1).mapNotNull { channel ->
val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: return@mapNotNull null
val fields = Wire.fields(Wire.bytes(service, 12) ?: return@mapNotNull null)
val name = Wire.bytes(fields, 1)?.let { String(it, Charsets.UTF_8) } ?: return@mapNotNull null
VendorService(id, name)
}
fun findBluetoothService(response: ByteArray): BluetoothService? {
for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue
val fields = Wire.fields(Wire.bytes(service, 6) ?: continue)
val address = Wire.bytes(fields, 1)?.let { String(it, Charsets.UTF_8) } ?: continue
// Packed as declared, though a car may send each method as its own varint.
val methods = fields.filter { it.number == 2 }.flatMap { f ->
f.bytes?.let(::packedVarints) ?: listOf(f.varint.toInt())
}
return BluetoothService(id, address, methods)
}
return null
}
const val STREAM_GUIDANCE = 1
const val STREAM_MEDIA = 3
const val CODEC_PCM = 1
const val CODEC_AAC_LC = 2
const val CODEC_AAC_LC_ADTS = 4
private val STREAMS = mapOf(0 to "NONE", 1 to "GUIDANCE", 2 to "SYSTEM_AUDIO", 3 to "MEDIA", 4 to "TELEPHONY")
@@ -29,12 +29,30 @@ object SessionReport {
UNKNOWN("unclassified"),
}
/**
* How far a session got, in the order a connection is built. The usage note's "Where
* connections fail" share sends the furthest one reached, so a failure says where it broke.
*/
enum class Stage(val key: String) {
USB("usb"), // the link opened; the car may have said nothing
VERSIONS("versions"), // the car said which protocol it speaks
TLS("tls"), // the secure link is up
ACCEPTED("accepted"), // the car accepted the certificate
CHANNELS("channels"), // the car listed its channels and said who it is
VIDEO("video"), // the car's screen is set up for Gearslip
STREAMING("streaming"), // video frames are reaching the car
}
private var appVersion = "?"
private var appCode = 0L
@Volatile private var startedAt = 0L
@Volatile var category = Category.NONE; private set
@Volatile private var failure = ""
@Volatile var stage = Stage.USB; private set
/** What the car or the phone said when it broke: a TLS alert, an auth status, a ByeBye reason, an errno. */
@Volatile private var failCode = ""
@Volatile private var failedAfter = 0L
@Volatile private var protocolVersion = ""
@Volatile private var certSource = ""
@Volatile private var certIssuer = ""
@@ -58,10 +76,17 @@ object SessionReport {
appCode = info?.longVersionCode ?: 0
}
fun begin() {
/** How this drive reached the car, and how its music did: "usb" or "wireless", "usb" or "bluetooth". */
@Volatile var link = ""; private set
@Volatile var audioMode = ""; private set
fun begin(wireless: Boolean) {
startedAt = System.currentTimeMillis()
link = if (wireless) "wireless" else "usb"
audioMode = if (app.seb3thehacker.gearslip.car.CarSettings.pipeAudio.value) "usb" else "bluetooth"
category = Category.NONE
failure = ""; protocolVersion = ""; certSource = ""; certIssuer = ""
stage = Stage.USB; failCode = ""; failedAfter = 0
tlsProtocol = ""; tlsCipher = ""; authStatus = null
headUnitInfo = null
carScreen = ""; carDpi = 0
@@ -69,34 +94,87 @@ object SessionReport {
printed = false
}
fun protocolVersion(major: Int, minor: Int) { protocolVersion = "$major.$minor" }
/** Moves the session's furthest stage on; never back. */
private fun reached(next: Stage) { if (next > stage) stage = next }
fun protocolVersion(major: Int, minor: Int) { protocolVersion = "$major.$minor"; reached(Stage.VERSIONS) }
fun certificate(source: String, issuer: String) { certSource = source; certIssuer = issuer }
fun tls(protocol: String, cipher: String) { tlsProtocol = protocol; tlsCipher = cipher }
fun auth(status: Int) { authStatus = status }
fun tls(protocol: String, cipher: String) { tlsProtocol = protocol; tlsCipher = cipher; reached(Stage.TLS) }
fun auth(status: Int) { authStatus = status; if (status == 0) reached(Stage.ACCEPTED) }
fun headUnit(info: String) { headUnit = info }
/** Who the car said it is, for the opt-in usage note; null until service discovery. */
@Volatile var headUnitInfo: ServiceDiscovery.HeadUnitInfo? = null
set(value) { field = value; if (value != null) reached(Stage.CHANNELS) }
/** The car screen's usable size once its margins are cropped, and its density; blank before video starts. */
@Volatile var carScreen = ""; private set
@Volatile var carDpi = 0; private set
fun carScreen(size: String, dpi: Int) { carScreen = size; carDpi = dpi }
fun carScreen(size: String, dpi: Int) { carScreen = size; carDpi = dpi; reached(Stage.VIDEO) }
fun profile(name: String?) { profile = name ?: "none matched" }
fun video(summary: String) { video = summary }
fun audio(summary: String) { audio = summary }
fun stream(summary: String) { lastStream = summary }
fun stream(summary: String) { lastStream = summary; reached(Stage.STREAMING) }
/** Records the first failure only: later ones are usually fallout from it. */
fun fail(category: Category, detail: String, state: String = "") {
/**
* Records the first failure only: later ones are usually fallout from it. [code] is the short
* thing the car or the phone said, like "auth -3" or "bye 1", for the usage note.
*/
fun fail(category: Category, detail: String, state: String = "", code: String = "") {
if (this.category != Category.NONE) return
this.category = category
failure = detail
lastState = state
failCode = code
failedAfter = if (startedAt == 0L) 0 else (System.currentTimeMillis() - startedAt) / 1000
}
fun fail(category: Category, detail: String, t: Throwable, state: String = "") =
fail(category, "$detail (${GearslipLog.rootCause(t)})", state)
fail(category, "$detail (${GearslipLog.rootCause(t)})", state, codeOf(t))
/**
* A short code from an exception, never its whole message: the TLS alert's name, the errno
* of a USB failure, or else the exception's class.
*/
private fun codeOf(t: Throwable): String {
var root = t
while (root.cause != null && root.cause !== root) root = root.cause!!
val text = "${t.message} ${root.message}"
Regex("(?i)alert[ _:]*\\(?([a-z_ ]{3,40}?)\\)?(?:\\s|$|[,;.])").find(text)?.let {
return "alert ${it.groupValues[1].trim().lowercase().replace(' ', '_')}"
}
Regex("\\b(E[A-Z]{2,12})\\b").find(text)?.let { return it.groupValues[1] }
return root.javaClass.simpleName.take(40)
}
/**
* One session for the usage note's "Where connections fail" share: how far it got, how and
* when it ended, and with which kind of certificate. Counts and codes only, nothing personal.
*/
fun sessionNote(): org.json.JSONObject {
val seconds = if (startedAt == 0L) 0 else (System.currentTimeMillis() - startedAt) / 1000
val source = certSource.lowercase()
return org.json.JSONObject()
.put("stage", stage.key)
.put("ended", category.name.lowercase())
.put("code", failCode)
.put("failed_at", if (category == Category.NONE) 0 else rounded(failedAfter))
.put("seconds", rounded(seconds))
.put("cert", when {
"imported" in source -> "imported"
"downloaded" in source -> "downloaded"
"self" in source -> "self-signed"
source.isEmpty() -> ""
else -> "other"
})
}
/** Seconds rounded so a drive's length can't be matched to anything: 5 s, then minutes, then 5 minutes. */
private fun rounded(s: Long): Long = when {
s < 60 -> (s + 2) / 5 * 5
s < 3600 -> (s + 30) / 60 * 60
else -> (s + 150) / 300 * 300
}
/** The report as plain text, for the log file and for anything that gets sent. */
fun render(): String {
@@ -145,7 +223,14 @@ object SessionReport {
category == Category.NONE -> "ended before video"
else -> category.label
}
runCatching { UsageStats.recordSession(context, headUnitInfo, protocolVersion, outcome, carScreen, carDpi) }
runCatching {
UsageStats.recordSession(context, headUnitInfo, protocolVersion, outcome, carScreen, carDpi, sessionNote(), link, audioMode)
}
if (outcome == "connected") runCatching {
app.seb3thehacker.gearslip.stats.WorkedPrompt.onDriveWorked(context, headUnitInfo, protocolVersion, carScreen, carDpi)
app.seb3thehacker.gearslip.ui.DonatePrompt.onDriveWorked(context)
}
runCatching { app.seb3thehacker.gearslip.update.UpdateChecker.onDriveEnded(context) }
}
GearslipLog.flush()
}
@@ -20,7 +20,7 @@ object SessionStatus {
Snapshot(
Phase.IDLE,
"Not connected",
"Plug the phone into the car's USB port. Gearslip opens by itself when the head unit connects.",
"Plug into the car's USB port, or connect wirelessly. Gearslip opens by itself when the car answers.",
),
)
@@ -23,6 +23,8 @@ class VideoSource(
private val height: Int,
private val frameRate: Int,
private val mode: Mode = Mode.TEST_CARD,
/** Over Wi-Fi: a fraction of the cable bitrate, so the link keeps up and sound isn't starved. */
private val wireless: Boolean = false,
private val onCodecConfig: (ByteArray) -> Unit,
private val onFrame: (data: ByteArray, presentationTimeUs: Long, keyFrame: Boolean) -> Unit,
) {
@@ -45,7 +47,10 @@ class VideoSource(
private var frameIndex = 0L
fun start() {
val bitRate = (width.toLong() * height * frameRate * BITS_PER_PIXEL).toInt().coerceIn(MIN_BIT_RATE, MAX_BIT_RATE)
val level = levelFor(width, height, frameRate)
val bitRate = defaultBitRate(width, height, frameRate, level).let {
if (wireless) (it * WIRELESS_SHARE).toInt().coerceAtMost(WIRELESS_MAX_BIT_RATE) else it
}
fun format(tuned: Boolean) = MediaFormat.createVideoFormat(MIME, width, height).apply {
setInteger(
MediaFormat.KEY_COLOR_FORMAT,
@@ -74,7 +79,7 @@ class VideoSource(
setFloat(MediaFormat.KEY_MAX_FPS_TO_ENCODER, frameRate.toFloat())
}
setInteger(MediaFormat.KEY_PROFILE, MediaCodecInfo.CodecProfileLevel.AVCProfileBaseline)
setInteger(MediaFormat.KEY_LEVEL, MediaCodecInfo.CodecProfileLevel.AVCLevel31)
setInteger(MediaFormat.KEY_LEVEL, level.codecLevel)
if (tuned) {
// Hints only: a car screen wants each frame now, not the best frame a moment later.
setInteger(MediaFormat.KEY_BITRATE_MODE, MediaCodecInfo.EncoderCapabilities.BITRATE_MODE_VBR)
@@ -98,7 +103,7 @@ class VideoSource(
codec = encoder
running = true
log.i("encoder started: ${width}x$height @ ${frameRate}fps, ${bitRate / 1000} kbps, H.264 Baseline, mode=$mode")
log.i("encoder started: ${width}x$height @ ${frameRate}fps, ${bitRate / 1000} kbps${if (wireless) " (wireless)" else ""}, H.264 Baseline level ${level.label}, mode=$mode")
worker = Thread {
runCatching { pump(encoder) }
@@ -223,10 +228,48 @@ class VideoSource(
const val TIMEOUT_US = 10_000L
const val SURFACE_TIMEOUT_US = 100_000L
/** Enough for sharp text and map lines; 800x480 at 30fps comes out at 4 Mbps. */
const val BITS_PER_PIXEL = 0.3
const val MIN_BIT_RATE = 4_000_000
const val MAX_BIT_RATE = 12_000_000
/**
* 800x480 at 30fps gets 8 Mbps: about the most that size can show, enough that a moving
* video in the browser stays sharp. Bigger screens get more, but less than in proportion,
* since large frames compress better: bitrate grows with pixels^0.75. 60fps gets 1.5x,
* because frames that close together are mostly alike.
*/
const val BASE_BIT_RATE = 8_000_000.0
const val BASE_PIXELS = 800.0 * 480
/** No car's decoder is asked for more than this, whatever its level allows. */
const val MAX_BIT_RATE = 25_000_000
/**
* The H.264 levels a car screen can need, smallest first: macroblocks a second each
* decodes, and the Baseline bitrate it must handle. The encoder declares the smallest
* that fits, and the bitrate stays within it, so a car decoder isn't asked for more than
* the level it was told about.
*/
enum class Level(val label: String, val codecLevel: Int, val macroblocksPerSecond: Int, val maxBitRate: Int) {
L31("3.1", MediaCodecInfo.CodecProfileLevel.AVCLevel31, 108_000, 14_000_000),
L32("3.2", MediaCodecInfo.CodecProfileLevel.AVCLevel32, 216_000, 20_000_000),
L4("4", MediaCodecInfo.CodecProfileLevel.AVCLevel4, 245_760, 20_000_000),
L42("4.2", MediaCodecInfo.CodecProfileLevel.AVCLevel42, 522_240, 50_000_000),
}
fun levelFor(width: Int, height: Int, frameRate: Int): Level {
val perSecond = ((width + 15) / 16) * ((height + 15) / 16) * frameRate
return Level.entries.firstOrNull { perSecond <= it.macroblocksPerSecond } ?: Level.L42
}
fun defaultBitRate(width: Int, height: Int, frameRate: Int, level: Level): Int {
val size = Math.pow(width.toDouble() * height / BASE_PIXELS, 0.75)
val rate = if (frameRate > 30) 1.5 else 1.0
return (BASE_BIT_RATE * size * rate).toInt().coerceAtMost(minOf(level.maxBitRate, MAX_BIT_RATE))
}
/**
* A car's Wi-Fi is shared, noisy and far slower than its nominal rate, and the sound rides
* the same link behind the video. Real Android Auto sends a few Mbps wirelessly; at the
* cable rate (15-20 Mbps for 720p) a laptop hotspot fell seconds behind and the music
* cut in and out. 720p30 comes out near 5.4 Mbps.
*/
const val WIRELESS_SHARE = 0.35
const val WIRELESS_MAX_BIT_RATE = 8_000_000
const val I_FRAME_INTERVAL_S = 5
private const val REPEAT_AFTER_US = 100_000L
@@ -0,0 +1,94 @@
package app.seb3thehacker.gearslip.audio
import android.media.MediaCodec
import android.media.MediaCodecInfo
import android.media.MediaFormat
/**
* Turns captured PCM into AAC-LC for a car whose music channel takes nothing else. Cars that
* offer PCM never get here: AAC costs a little delay and quality, and every car that works today
* is a PCM car.
*
* Android's encoder hands back raw access units, which is what AUDIO_AAC_LC expects. For
* AUDIO_AAC_LC_ADTS each unit gets the 7-byte ADTS header in front, so the car can find the
* frame edges itself. The encoder's first output is its configuration, not audio, and is skipped.
*/
internal class AacEncoder(private val sampleRate: Int, private val channels: Int, private val adts: Boolean) {
private val codec: MediaCodec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC).apply {
val format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRate, channels).apply {
setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC)
setInteger(MediaFormat.KEY_BIT_RATE, if (channels == 1) 96_000 else 192_000)
setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 16 * 1024)
}
configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
start()
}
private val info = MediaCodec.BufferInfo()
private var samplesIn = 0L
/** Encodes [size] bytes of 16-bit PCM and returns the AAC frames that came out, oldest first. */
fun encode(pcm: ByteArray, size: Int): List<ByteArray> {
var offset = 0
val out = ArrayList<ByteArray>()
while (offset < size) {
val index = codec.dequeueInputBuffer(INPUT_WAIT_US)
if (index < 0) {
drain(out)
continue
}
val buffer = codec.getInputBuffer(index) ?: continue
val n = minOf(buffer.remaining(), size - offset)
buffer.put(pcm, offset, n)
val presentationUs = samplesIn * 1_000_000L / sampleRate
codec.queueInputBuffer(index, 0, n, presentationUs, 0)
samplesIn += n / (2 * channels)
offset += n
drain(out)
}
return out
}
private fun drain(out: MutableList<ByteArray>) {
while (true) {
val index = codec.dequeueOutputBuffer(info, 0)
if (index < 0) return
if (info.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG == 0 && info.size > 0) {
val buffer = codec.getOutputBuffer(index)
if (buffer != null) {
val frame = ByteArray(info.size)
buffer.position(info.offset)
buffer.get(frame)
out.add(if (adts) adtsHeader(frame.size) + frame else frame)
}
}
codec.releaseOutputBuffer(index, false)
}
}
/** ADTS for one AAC-LC frame: sync word, no CRC, and the frame's length including this header. */
private fun adtsHeader(payload: Int): ByteArray {
val length = payload + 7
val freq = FREQUENCIES.indexOf(sampleRate).let { if (it < 0) 3 else it }
val profile = 1 // AAC-LC is object type 2, written as type minus one
return byteArrayOf(
0xFF.toByte(),
0xF1.toByte(),
((profile shl 6) or (freq shl 2) or (channels shr 2)).toByte(),
(((channels and 3) shl 6) or (length shr 11)).toByte(),
((length shr 3) and 0xFF).toByte(),
(((length and 7) shl 5) or 0x1F).toByte(),
0xFC.toByte(),
)
}
fun release() {
runCatching { codec.stop() }
runCatching { codec.release() }
}
private companion object {
const val INPUT_WAIT_US = 10_000L
val FREQUENCIES = listOf(96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350)
}
}
@@ -16,6 +16,7 @@ import android.media.AudioRecord
import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager
import android.os.IBinder
import android.os.Process
import app.seb3thehacker.gearslip.GearslipLog
import kotlin.math.abs
@@ -147,6 +148,9 @@ class AudioCaptureService : Service() {
reading = true
thread = Thread({
// A busy phone (a map app drawing, the video encoder) would otherwise starve this
// loop, and the car hears every late read as a skip.
Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO)
val buffer = ByteArray(CHUNK_BYTES)
var peak = 0
var heardAt = System.currentTimeMillis()
@@ -22,6 +22,15 @@ class AudioLink(
private val service: ServiceDiscovery.AudioService,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
private val sendControl: (messageId: Int, body: ByteArray) -> Unit,
/** Log name: "audio" for the media sink, "guidance" for the directions sink. */
private val tag: String = "audio",
private val sessionId: Int = SESSION_ID,
/** Media asks the car for focus; directions just start, and the car ducks music under them. */
private val asksFocus: Boolean = true,
private val onReady: (AudioLink) -> Unit = { CarAudio.attach(it) },
private val onClose: (AudioLink) -> Unit = { CarAudio.detach(it) },
/** The car took the sound back after giving it, as when the driver switches to the radio. */
private val onTakenBack: () -> Unit = {},
) : CarAudio.Link {
private var configIndex = 0
@@ -33,11 +42,15 @@ class AudioLink(
// The Uconnect asks for 4 on its video channel; assume the same until its Config says otherwise.
@Volatile private var maxUnacked = 4
@Volatile private var lastAckAt = 0L
private var stalls = 0
private var chunks = 0
private var bytes = 0L
private var dropped = 0
@Volatile private var configTimer: Thread? = null
private val ackLock = Object()
/** Only for a sink that takes AAC and nothing else; PCM sinks get the capture as it is. */
@Volatile private var aac: AacEncoder? = null
private val aacSink get() = service.codecType == ServiceDiscovery.CODEC_AAC_LC || service.codecType == ServiceDiscovery.CODEC_AAC_LC_ADTS
private val config get() = service.configs.getOrNull(configIndex) ?: service.configs.firstOrNull()
override val sampleRate get() = config?.sampleRate ?: 48_000
@@ -46,28 +59,29 @@ class AudioLink(
val channelId get() = service.serviceId
fun opened(status: Int) {
GearslipLog.i("audio: <- ChannelOpenResponse (media audio): status=$status")
GearslipLog.i("$tag: <- ChannelOpenResponse: status=$status")
if (status != 0) {
GearslipLog.w("audio: the head unit refused the media audio channel (status $status)")
GearslipLog.w("$tag: the head unit refused the $tag channel (status $status)")
return
}
// AVChannelSetupRequest { required uint32 config_index = 1 }. This field is an index into
// the sink's own audio_configs, not a codec id - sending the codec type asks for a config
// the sink does not have, and it answers with a status that never reaches READY.
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, 0L))
GearslipLog.i("audio: -> Setup(config_index=0) on channel ${service.serviceId}, codec ${service.codecName}")
// Setup { required MediaCodecType type = 1 }: the codec the car advertised for this sink
// (aasdk's current protos, and LIVI reads it as the codec). Gearslip used to send 0 here,
// read from an older proto as a config index; 0 is no codec at all, and the 2024 VW
// answered UNEXPECTED_MESSAGE (id 255), never acked a chunk and played nothing.
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, service.codecType.toLong()))
GearslipLog.i("$tag: -> Setup(codec=${service.codecName}) on channel ${service.serviceId}")
armConfigFallback()
}
/**
* The 2018 Uconnect answers our Setup with a two-byte message we cannot identify (id 255, no
* body) rather than the AVChannelSetupResponse the video channel gets, so the sink never
* reaches READY and no audio is ever sent.
* The 2018 Uconnect, a Dacia ULC 4.5 and the 2024 VW answered the old Setup (codec 0) with
* UNEXPECTED_MESSAGE (id 255, no body) rather than a Config, so the sink never reached READY.
* [MSG_SETUP_ANSWER] still takes that as the go-ahead, in case a car rejects the real codec too.
*
* Service discovery already told us everything that message would have: the sink is PCM at
* 48000Hz stereo and it has exactly one configuration. So rather than wait forever for a
* message that is not coming, assume the advertised configuration and carry on. If that guess
* is wrong the head unit simply will not ack, which the write path already reports.
* Service discovery already told us everything a Config would have: the sink's rate, channels
* and its one configuration. So for a car that answers nothing at all, wait a moment and then
* assume the advertised configuration. If that guess is wrong the head unit simply will not
* ack, which the write path already reports.
*/
private fun armConfigFallback() {
val timer = Thread({
@@ -76,19 +90,25 @@ class AudioLink(
} catch (e: InterruptedException) {
return@Thread
}
if (ready) return@Thread
GearslipLog.w(
"audio: no usable Config after ${CONFIG_WAIT_MS}ms - falling back to the sink the " +
"head unit advertised (${sampleRate}Hz x$channels, config_index=0, max_unacked=$maxUnacked)",
)
configIndex = 0
ready = true
CarAudio.attach(this)
useAdvertisedSink("no usable Config after ${CONFIG_WAIT_MS}ms")
}, "gearslip-audio-config").apply { isDaemon = true }
configTimer = timer
timer.start()
}
@Synchronized
private fun useAdvertisedSink(why: String) {
if (ready) return
configTimer?.interrupt()
GearslipLog.i(
"$tag: $why - using the sink the head unit advertised " +
"(${sampleRate}Hz x$channels, config_index=0, max_unacked=$maxUnacked)",
)
configIndex = 0
ready = true
onReady(this)
}
fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> opened(Protobuf.readInt32Field(body, 1) ?: 0)
@@ -98,17 +118,17 @@ class AudioLink(
val status = Wire.varint(fields, 1)?.toInt()
val window = Wire.varint(fields, 2)?.toInt()
val indices = fields.filter { it.number == 3 && it.wireType == 0 }.map { it.varint.toInt() }
GearslipLog.i("audio: <- Config status=$status max_unacked=$window indices=$indices")
GearslipLog.i("$tag: <- Config status=$status max_unacked=$window indices=$indices")
if (window != null) maxUnacked = maxOf(1, window)
configIndex = indices.firstOrNull() ?: 0
if (status == STATUS_READY) {
configTimer?.interrupt()
ready = true
GearslipLog.i("audio: sink ready - ${sampleRate}Hz x$channels ${config?.bits}bit")
CarAudio.attach(this)
GearslipLog.i("$tag: sink ready - ${sampleRate}Hz x$channels ${config?.bits}bit")
onReady(this)
} else {
GearslipLog.w(
"audio: the sink is not ready (status=$status, 1=WAIT 2=READY) - " +
"$tag: the sink is not ready (status=$status, 1=WAIT 2=READY) - " +
"nothing will be sent until it is",
)
}
@@ -123,9 +143,11 @@ class AudioLink(
}
}
// Whatever the head unit answers Setup with, we need its bytes to identify it.
else -> {
GearslipLog.i("audio: <- unhandled message id=$messageId (${body.size} bytes)")
// Whatever else the head unit answers Setup with, we need its bytes to identify it.
else -> if (messageId == MSG_SETUP_ANSWER && body.isEmpty()) {
useAdvertisedSink("<- empty Setup answer (id $messageId)")
} else {
GearslipLog.i("$tag: <- unhandled message id=$messageId (${body.size} bytes)")
GearslipLog.hex(" audio msg", body, limit = 64)
}
}
@@ -138,17 +160,21 @@ class AudioLink(
*/
fun onFocus(state: Int) {
val name = FOCUS_STATES[state] ?: "unknown($state)"
GearslipLog.i("audio: <- AudioFocusResponse state=$name")
focusGranted = state == FOCUS_STATE_GAIN || state == FOCUS_STATE_GAIN_TRANSIENT
GearslipLog.i("$tag: <- AudioFocusResponse state=$name")
val had = focusGranted
focusGranted = state == FOCUS_STATE_GAIN || state == FOCUS_STATE_GAIN_TRANSIENT ||
state == FOCUS_STATE_GAIN_MEDIA_ONLY
if (focusGranted) startStream() else stop()
// A ducking request leaves the music playing; a loss means another source has the speakers.
if (had && (state == FOCUS_STATE_LOSS || state == FOCUS_STATE_LOSS_TRANSIENT)) onTakenBack()
}
override fun start() {
if (!ready || started) return
if (!focusRequested) {
if (asksFocus && !focusRequested) {
focusRequested = true
sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_GAIN.toLong()))
GearslipLog.i("audio: -> AudioFocusRequest(GAIN); waiting for the head unit to hand the sink over")
GearslipLog.i("$tag: -> AudioFocusRequest(GAIN); waiting for the head unit to hand the sink over")
}
// Some head units grant the sink silently and never answer. Do not let that be the end of
// it: start anyway, and let the acks say whether anything is being taken.
@@ -159,12 +185,18 @@ class AudioLink(
if (!ready || started) return
sendOnChannel(
MSG_START,
Protobuf.varintField(1, SESSION_ID.toLong()) + Protobuf.varintField(2, configIndex.toLong()),
Protobuf.varintField(1, sessionId.toLong()) + Protobuf.varintField(2, configIndex.toLong()),
)
inFlight = 0
lastAckAt = System.currentTimeMillis()
if (aacSink && aac == null) {
aac = runCatching { AacEncoder(sampleRate, channels, adts = service.codecType == ServiceDiscovery.CODEC_AAC_LC_ADTS) }
.onSuccess { GearslipLog.i("$tag: the car takes ${service.codecName} only - encoding to AAC") }
.onFailure { GearslipLog.e("$tag: couldn't start the AAC encoder", it) }
.getOrNull()
}
started = true
GearslipLog.i("audio: -> Start(session=$SESSION_ID, config=$configIndex) - streaming ${sampleRate}Hz x$channels")
GearslipLog.i("$tag: -> Start(session=$sessionId, config=$configIndex) - streaming ${sampleRate}Hz x$channels")
}
override fun stop() {
@@ -172,18 +204,39 @@ class AudioLink(
started = false
synchronized(ackLock) { ackLock.notifyAll() }
runCatching {
sendOnChannel(MSG_STOP, Protobuf.varintField(1, SESSION_ID.toLong()))
sendOnChannel(MSG_STOP, Protobuf.varintField(1, sessionId.toLong()))
if (focusRequested) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_RELEASE.toLong()))
}
focusRequested = false
focusGranted = false
GearslipLog.i("audio: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks / $bytes bytes")
aac?.release()
aac = null
GearslipLog.i("$tag: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks / $bytes bytes")
chunks = 0
bytes = 0
}
override fun write(pcm: ByteArray, size: Int) {
if (!started) return
val encoder = aac
if (encoder == null) {
send(pcm.copyOf(size))
return
}
val frames = try {
encoder.encode(pcm, size)
} catch (e: Exception) {
// A codec that dies mid-drive leaves the car silent, as it was before AAC; log it once.
GearslipLog.e("$tag: AAC encoding failed - no more audio this drive", e)
aac = null
started = false
return
}
frames.forEach(::send)
}
/** One Data message, once the car has room for it. */
private fun send(payload: ByteArray) {
// Wait for the car to catch up rather than throw the chunk away: a dropped chunk is a gap
// the driver hears, while a short wait is soaked up by the capture buffer behind us.
synchronized(ackLock) {
@@ -191,7 +244,11 @@ class AudioLink(
val waited = System.currentTimeMillis() - lastAckAt
if (waited > STALL_MS) {
// A lost ack must not silence the stream for good; after a stall assume the window cleared.
GearslipLog.w("audio: no ack for ${STALL_MS}ms with $inFlight in flight - reopening the window")
// A car that never acks (the 2024 VW, the Bolt) hits this every chunk: say so
// once, then now and then, so the rest of the log stays readable.
if (stalls++ % 500 == 0) {
GearslipLog.w("$tag: no ack for ${STALL_MS}ms with $inFlight in flight - reopening the window ($stalls times)")
}
inFlight = 0
break
}
@@ -206,20 +263,20 @@ class AudioLink(
val micros = System.nanoTime() / 1_000
for (i in 0 until 8) timestamp[i] = ((micros shr ((7 - i) * 8)) and 0xFF).toByte()
try {
sendOnChannel(MSG_DATA, timestamp + pcm.copyOf(size))
sendOnChannel(MSG_DATA, timestamp + payload)
} catch (e: java.io.IOException) {
GearslipLog.e("audio: the link died mid-stream", e)
GearslipLog.e("$tag: the link died mid-stream", e)
started = false
return
}
synchronized(ackLock) { inFlight++ }
chunks++
bytes += size
bytes += payload.size
// The first chunk proves the whole path; after that, an occasional line is enough to show
// the car is still taking audio and to catch a window that has quietly closed.
if (chunks == 1 || chunks % CHUNK_LOG_EVERY == 0) {
GearslipLog.i(
"audio: -> $chunks chunks / $bytes bytes, inFlight=$inFlight/$maxUnacked, " +
"$tag: -> $chunks chunks / $bytes bytes, inFlight=$inFlight/$maxUnacked, " +
"dropped=$dropped, focus=${if (focusGranted) "granted" else "not granted"}",
)
}
@@ -230,7 +287,7 @@ class AudioLink(
configTimer = null
stop()
ready = false
CarAudio.detach(this)
onClose(this)
}
companion object {
@@ -243,18 +300,24 @@ class AudioLink(
private const val MSG_STOP = 32770
private const val MSG_CONFIG = 32771
private const val MSG_ACK = 32772
private const val MSG_SETUP_ANSWER = 255
private const val STATUS_READY = 2
private const val FOCUS_GAIN = 1
private const val FOCUS_RELEASE = 4
private const val FOCUS_STATE_GAIN = 1
private const val FOCUS_STATE_GAIN_TRANSIENT = 2
private const val FOCUS_STATE_GAIN_MEDIA_ONLY = 6
private const val FOCUS_STATE_LOSS = 3
private const val FOCUS_STATE_LOSS_TRANSIENT = 5
private const val SESSION_ID = 2
private const val STALL_MS = 2_000L
private const val CHUNK_LOG_EVERY = 200 // about every four seconds
private const val CONFIG_WAIT_MS = 2_500L
private const val CONFIG_WAIT_MS = 1_000L
// AudioFocusStateNotification.AudioFocusState in aasdk.
private val FOCUS_STATES = mapOf(
1 to "GAIN", 2 to "GAIN_TRANSIENT", 3 to "LOSS", 4 to "LOSS_TRANSIENT",
1 to "GAIN", 2 to "GAIN_TRANSIENT", 3 to "LOSS", 4 to "LOSS_TRANSIENT_CAN_DUCK",
5 to "LOSS_TRANSIENT", 6 to "GAIN_MEDIA_ONLY", 7 to "GAIN_TRANSIENT_GUIDANCE_ONLY",
)
}
}
@@ -100,6 +100,26 @@ object CarAudio {
_capture.value = Capture.DENIED
}
private val _micBlocked = MutableStateFlow(false)
/** The microphone permission is off for good: Android refuses without asking, so say why. */
val micBlocked: StateFlow<Boolean> = _micBlocked.asStateFlow()
fun micBlocked() {
_micBlocked.value = true
denied()
}
/** The driver turned the permission back on mid-drive: ask for capture again. */
fun micAllowed() {
if (!_micBlocked.value) return
_micBlocked.value = false
if (_capture.value == Capture.DENIED) {
_capture.value = Capture.OFF
if (wanted) request()
}
}
internal fun captureStarted() {
_capture.value = Capture.ON
link?.start()
@@ -12,17 +12,21 @@ import java.util.concurrent.TimeUnit
*
* -> CHANNEL_OPEN_REQUEST <- CHANNEL_OPEN_RESPONSE
* -> Setup { config_index } <- Config
* -> VoiceSessionNotification { START } (control channel)
* -> MicrophoneRequest { open } <- MicrophoneResponse { status, session_id }
* <- Data { 8-byte timestamp, PCM }... -> Ack { session_id, ack }
* -> MicrophoneRequest { open = false }
* -> VoiceSessionNotification { END }
*
* The channel opens with the session, but the car only records between [start] and [stop].
* The channel opens with the session, but the car only records between [start] and [stop], and
* only inside a voice session: the 2024 VW answered a bare MicrophoneRequest with status -7.
* Speech wants 16 kHz mono; a car that records at a multiple of that is averaged down to it, and
* one that doesn't isn't used, so voice falls back to the phone's microphone.
*/
class CarMic(
private val service: ServiceDiscovery.AudioService,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
private val voiceSession: (start: Boolean) -> Unit,
) {
val channelId get() = service.serviceId
@@ -43,7 +47,8 @@ class CarMic(
val status = Protobuf.readInt32Field(body, 1) ?: 0
GearslipLog.i("mic: <- ChannelOpenResponse: status=$status")
if (status != 0) return
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, 0L))
// Setup { type = the codec }, as for audio: 0 got UNEXPECTED_MESSAGE from the VW.
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, service.codecType.toLong()))
ready = usable
if (!usable) {
GearslipLog.w(
@@ -97,6 +102,7 @@ class CarMic(
chunks.clear()
packets = 0
listening = true
voiceSession(true)
sendOnChannel(MSG_MIC_REQUEST, Protobuf.varintField(1, 1L) + Protobuf.varintField(4, 1L))
GearslipLog.i("mic: -> MicrophoneRequest(open)")
return true
@@ -109,7 +115,8 @@ class CarMic(
if (!listening) return
listening = false
sendOnChannel(MSG_MIC_REQUEST, Protobuf.varintField(1, 0L))
GearslipLog.i("mic: -> MicrophoneRequest(close) after $packets packets")
voiceSession(false)
GearslipLog.i("mic: -> MicrophoneRequest(close) after $packets packets, voice session ended")
}
companion object {
@@ -1,10 +1,14 @@
package app.seb3thehacker.gearslip.car
import android.content.Context
import app.seb3thehacker.gearslip.media.CarMedia
import app.seb3thehacker.gearslip.speech.VoiceFocus
import java.text.DateFormat
import java.util.Date
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeoutOrNull
/**
* Turns what [CarAssistant] heard into an action plus something to say back.
@@ -17,9 +21,16 @@ import kotlinx.coroutines.withContext
object AssistantCommands {
suspend fun handle(context: Context, navigator: CarNavigator, frame: CarEnvironment.Frame, heardRaw: String): String {
val heard = heardRaw.lowercase().trim()
val heard = natural(heardRaw)
VoiceHelp.hide()
if (heard in HELP_WORDS || heard.startsWith("what can i say") || heard.startsWith("what can you do")) {
VoiceHelp.show()
return "Here's what you can say."
}
if (heard.matches(Regex("(fuck|f\\*+) (you|off)( too)?"))) return "Fuck you too."
navigateCommand(context, navigator, frame, heard)?.let { return it }
playSearchCommand(context, navigator, heard)?.let { return it }
mediaCommand(heard)?.let { return it }
navCommand(navigator, heard)?.let { return it }
infoCommand(heard)?.let { return it }
@@ -29,6 +40,47 @@ object AssistantCommands {
return "I don't know how to do that yet."
}
/**
* What was said, minus the padding people wrap a command in: "hey", "can you", "I want to",
* "please", "thanks". Each command below then only has to know its own words.
*/
private fun natural(raw: String): String {
var t = raw.lowercase().replace(Regex("[.,!?]"), " ").replace(Regex("\\s+"), " ").trim()
var changed = true
while (changed) {
changed = false
for (lead in LEAD_FILLERS) if (t.startsWith("$lead ")) { t = t.removePrefix("$lead ").trim(); changed = true }
for (tail in TAIL_FILLERS) if (t.endsWith(" $tail")) { t = t.removeSuffix(" $tail").trim(); changed = true }
}
return t
}
/** The part after the first of [triggers] that [text] starts with, or null. */
private fun after(text: String, triggers: List<String>): String? =
triggers.firstOrNull { text.startsWith(it) }?.let { text.removePrefix(it).trim() }
private suspend fun allEntries(context: Context): List<Entry> {
val vehicleData = CarSettings.experimentalFeaturesEnabled.value && CarSettings.carSensorsEnabled.value
val builtIn = BuiltInApps.entries.filter { it.builtIn !== BuiltInApps.VehicleData || vehicleData }
return builtIn + (LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) })
}
/**
* Best label match: exact, then a label starting with [name], then one containing it. Spaces
* and punctuation don't count, since the recognizer may hear "MapQuest" as "map quest".
*/
private fun <T> bestMatch(items: List<T>, name: String, label: (T) -> String): T? {
val n = squash(name.removePrefix("the ").removePrefix("my ").removeSuffix(" app"))
if (n.isEmpty()) return null
items.firstOrNull { squash(label(it)) == n }?.let { return it }
// Loose matches need a real word: "pizza in a" mustn't pick "Organic Maps" for its "a".
if (n.length < 3) return null
return items.firstOrNull { squash(label(it)).startsWith(n) }
?: items.firstOrNull { squash(label(it)).contains(n) }
}
private fun squash(s: String) = s.lowercase().filter { it.isLetterOrDigit() }
/**
* "navigate to X [with|using|on|via Y]" - Y picks the nav app by (partial) label; without it,
* whichever nav app was used last, or the only one installed, or the first one found. Starts
@@ -42,29 +94,49 @@ object AssistantCommands {
frame: CarEnvironment.Frame,
heard: String,
): String? {
val trigger = NAVIGATE_TRIGGERS.firstOrNull { heard.startsWith(it) } ?: return null
var destination = heard.removePrefix(trigger).trim()
// "use Organic Maps to get to X", "open OsmAnd and navigate to X": the app comes first.
var text = heard
var leadingApp: String? = null
Regex("^(?:use|using|with|open|in|on) (.+?) (?:to |and )(.+)$").find(text)?.let { m ->
if (after(m.groupValues[2], NAVIGATE_TRIGGERS) != null) {
leadingApp = m.groupValues[1]
text = m.groupValues[2]
}
}
var destination = after(text, NAVIGATE_TRIGGERS) ?: return null
if (destination.isBlank()) return "Where to?"
var appName: String? = null
for (sep in listOf(" with ", " using ", " on ", " via ")) {
val at = destination.lastIndexOf(sep)
if (at > 0) {
appName = destination.substring(at + sep.length).trim()
val entries = allEntries(context)
// "go to Spotify" means the app, and "go to settings" the screen; "go to Target" is a place.
val asApp = destination.removePrefix("the ").removeSuffix(" app")
if (text.startsWith("go to ") &&
(asApp in SCREEN_WORDS || asApp in APP_NICKNAMES || entries.any { it.label.lowercase() == asApp })
) {
return null
}
val navApps = entries.filter { it.template?.isNavigation == true }
if (navApps.isEmpty()) return "You don't have any navigation apps installed."
// "X with Organic Maps", "X on OsmAnd": split only when the end names a nav app, so a
// place like "Starbucks on Main Street" stays whole.
var chosenApp = leadingApp?.let { name -> bestMatch(navApps, name) { it.label } ?: return "I couldn't find a navigation app called $name." }
if (chosenApp == null) {
for (sep in listOf(" with ", " using ", " on ", " via ", " in ", " through ")) {
val at = destination.lastIndexOf(sep)
if (at <= 0) continue
val app = bestMatch(navApps, destination.substring(at + sep.length)) { it.label } ?: continue
chosenApp = app
destination = destination.substring(0, at).trim()
break
}
}
// "the nearest gas station" -> "gas station": the map app already searches around the car
// and AutoRoute picks the nearest result, and Organic Maps took "nearest" as part of the name.
val (place, nearby) = plainPlace(destination)
destination = place
if (destination.isBlank()) return "Where to?"
val entries = LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) }
val navApps = entries.filter { it.template?.isNavigation == true }
if (navApps.isEmpty()) return "You don't have any navigation apps installed."
val requestedApp = appName
val chosen = when {
requestedApp != null -> navApps.firstOrNull { it.label.lowercase().contains(requestedApp) }
?: return "I couldn't find a navigation app called $requestedApp."
val chosen = chosenApp ?: when {
navApps.size == 1 -> navApps.first()
else -> navApps.firstOrNull { it.componentId == CarSettings.lastNav.value } ?: navApps.first()
}
@@ -72,29 +144,104 @@ object AssistantCommands {
CarServices.connectNav(template, frame, destination)
navigator.home()
return "Navigating to $destination with ${chosen.label}."
return if (nearby) "Finding the nearest $destination with ${chosen.label}." else "Navigating to $destination with ${chosen.label}."
}
/** [destination] without "a", "the nearest", "near me" and the like, and whether any of that said "close by". */
private fun plainPlace(destination: String): Pair<String, Boolean> {
var t = destination
var nearby = false
var changed = true
while (changed) {
changed = false
for (lead in PLACE_LEADS) if (t.startsWith(lead)) {
t = t.removePrefix(lead).trim(); changed = true
if (lead.trim() !in ARTICLES) nearby = true
}
for (tail in PLACE_TAILS) if (t.endsWith(tail)) { t = t.removeSuffix(tail).trim(); changed = true; nearby = true }
}
return t to nearby
}
/**
* "play X" with something after it: a song, artist, album or playlist, asked of the music app
* that's open through its own search ([CarMedia.search]). A bare "play", or "play music",
* falls through to resume.
*/
private suspend fun playSearchCommand(context: Context, navigator: CarNavigator, heard: String): String? {
var query = after(heard, PLAY_TRIGGERS) ?: return null
query = query.removePrefix("some ").trim()
if (query.isEmpty() || query in RESUME_WORDS) return null
val media = CarServices.media
// "play Daft Punk on Metrolist": switch to that app first, if it's one we know.
for (sep in listOf(" on ", " in ", " from ", " with ", " using ")) {
val at = query.lastIndexOf(sep)
if (at <= 0) continue
val apps = withContext(Dispatchers.IO) { app.seb3thehacker.gearslip.media.MediaCatalog.installed(context) }
val wanted = bestMatch(apps, query.substring(at + sep.length)) { it.label } ?: continue
query = query.substring(0, at).trim()
if (query.isEmpty() || query in RESUME_WORDS) {
CarServices.openMedia(wanted, autoplay = true)
navigator.media()
return "Opening ${wanted.label}."
}
if (CarServices.mediaApp.value?.component != wanted.component) {
CarServices.openMedia(wanted)
navigator.media()
// Give it a moment to connect before asking it to search.
withTimeoutOrNull(APP_CONNECT_MS) {
while (media.appLabel == null || (!media.canSearchLibrary && !media.now.value.canSearch)) delay(200)
}
}
break
}
// No music app open, or its connection is gone (the app was closed): use the one played
// last, the way "play X" works on Android Auto.
val phase = media.phase.value
if (media.appLabel == null || phase == CarMedia.Phase.IDLE || phase == CarMedia.Phase.REJECTED) {
if (!CarServices.reopenLastMedia()) return "Open a music app first, then ask again."
navigator.media()
}
withTimeoutOrNull(APP_CONNECT_MS) {
while (media.phase.value != CarMedia.Phase.READY || media.appLabel == null ||
(!media.canSearchLibrary && !media.now.value.canSearch)
) delay(200)
}
val app = media.appLabel ?: return "Your music app didn't open in time. Try again."
if (!media.canSearchLibrary && !media.now.value.canSearch) {
return "$app ignores voice searches. Pick something on its screen instead."
}
media.playBestMatch(query)
return "Looking for $query on $app."
}
private fun mediaCommand(heard: String): String? {
val media = CarServices.media
return when {
"pause" in heard || heard.startsWith("stop") -> {
if (media.now.value.playing) media.togglePlay()
"pause" in heard || heard.startsWith("stop") || heard == "quiet" || heard == "shut up" -> {
media.pause()
VoiceFocus.afterRelease { media.pause() }
"Paused."
}
"resume" in heard || (heard.startsWith("play") && "playing" !in heard) -> {
"resume" in heard || "unpause" in heard || heard in RESUME_PHRASES ||
(heard.startsWith("play") && "playing" !in heard) -> {
if (!media.now.value.playing) media.togglePlay()
"Playing."
}
"next" in heard || "skip" in heard -> {
"next" in heard || "skip" in heard || heard.startsWith("change the song") -> {
media.next()
"Skipping."
}
"previous" in heard || "last song" in heard || "back song" in heard -> {
"previous" in heard || "last song" in heard || "back song" in heard || "go back a song" in heard ||
"play that again" in heard || "replay" in heard -> {
media.previous()
"Playing the previous track."
}
"what" in heard && "playing" in heard -> {
("what" in heard && "playing" in heard) || "what song" in heard || "who sings" in heard ||
"who's singing" in heard || "who is singing" in heard || "what's this song" in heard ||
"name of this song" in heard -> {
val now = media.now.value
if (now.isActive) listOf(now.title, now.artist).filter { it.isNotBlank() }.joinToString(" by ")
else "Nothing's playing."
@@ -104,11 +251,19 @@ object AssistantCommands {
}
private fun navCommand(navigator: CarNavigator, heard: String): String? = when {
"go home" in heard || heard == "home" -> {
heard == "go back" || heard == "back" -> {
navigator.back()
"Going back."
}
heard in EXIT_WORDS -> {
BuiltInApps.Exit.open(navigator)
"Back to the car's screen."
}
"go home" in heard || heard == "home" || heard == "home screen" || "go to the home screen" in heard -> {
navigator.home()
"Heading home."
}
"open apps" in heard || heard == "apps" || "app list" in heard -> {
"open apps" in heard || heard == "apps" || "app list" in heard || "all apps" in heard || "my apps" in heard || heard == "go to apps" -> {
navigator.apps()
"Here are your apps."
}
@@ -128,9 +283,10 @@ object AssistantCommands {
}
private fun infoCommand(heard: String): String? = when {
"what time" in heard || "what's the time" in heard ->
"what time" in heard || "what's the time" in heard || heard == "time" || "the time" in heard ->
"It's ${DateFormat.getTimeInstance(DateFormat.SHORT).format(Date())}."
"weather" in heard -> {
"weather" in heard || "going to rain" in heard || "gonna rain" in heard || "how hot" in heard ||
"how cold" in heard || "temperature outside" in heard -> {
val data = (Weather.state.value as? WeatherState.Ready)?.data
if (data != null) "${data.temp}${data.tempUnit} and ${Weather.describe(data.code)} in ${data.place}."
else "I don't have the weather yet."
@@ -139,12 +295,11 @@ object AssistantCommands {
}
private suspend fun callCommand(context: Context, heard: String): String? {
val name = when {
heard.startsWith("call ") -> heard.removePrefix("call ")
heard.startsWith("phone ") -> heard.removePrefix("phone ")
heard.startsWith("dial ") -> heard.removePrefix("dial ")
else -> return null
}.trim()
val name = (
after(heard, CALL_TRIGGERS)
?: Regex("^give (.+) a (?:call|ring)$").find(heard)?.groupValues?.get(1)
?: return null
).replace(Regex(" (?:on|at) (?:their |his |her )?(?:mobile|cell|phone|home|work)$"), "").trim()
if (name.isBlank()) return "Who should I call?"
if (!hasCallPermissions(context)) return "Gearslip needs the phone and contacts permissions first, from the phone screen."
@@ -162,21 +317,69 @@ object AssistantCommands {
frame: CarEnvironment.Frame,
heard: String,
): String? {
val name = when {
heard.startsWith("open ") -> heard.removePrefix("open ")
heard.startsWith("launch ") -> heard.removePrefix("launch ")
heard.startsWith("start ") -> heard.removePrefix("start ")
else -> return null
}.trim()
val name = (after(heard, OPEN_TRIGGERS) ?: return null)
.removePrefix("the ").removePrefix("my ").removeSuffix(" app").trim()
if (name.isBlank()) return null
val wanted = APP_NICKNAMES[name] ?: name
val entries = LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) }
val match = entries.firstOrNull { it.label.lowercase().contains(name) }
// Gearslip's own apps come first in the list, so "phone" opens its dialer, not the phone's.
val match = bestMatch(allEntries(context), wanted) { it.label }
?: return "I couldn't find an app called $name."
launchEntry(match, navigator, frame)
return "Opening ${match.label}."
}
private val NAVIGATE_TRIGGERS = listOf("navigate to ", "directions to ", "take me to ", "drive to ")
private val EXIT_WORDS = setOf("exit", "exit gearslip", "close gearslip", "quit", "quit gearslip", "open exit")
/** Other names people use for Gearslip's own apps. */
private val APP_NICKNAMES = mapOf(
"web" to "browser", "internet" to "browser", "the web" to "browser", "web browser" to "browser",
"dialer" to "phone", "the dialer" to "phone", "keypad" to "phone",
"screen share" to "screen sharing", "screen mirroring" to "screen sharing", "mirroring" to "screen sharing",
)
private val NAVIGATE_TRIGGERS = listOf(
"navigate to ", "navigate me to ", "directions to ", "get directions to ", "give me directions to ",
"take me to ", "drive to ", "drive me to ", "get me to ", "go to ", "head to ", "route to ",
"how do i get to ", "show me the way to ", "find a route to ", "navigation to ", "get to ",
// Last, so "navigate to" wins: the recognizer sometimes drops the "to".
"navigate me ", "navigate ", "find me the nearest ", "find the nearest ", "find the closest ",
)
private val ARTICLES = setOf("a", "an", "the", "some")
// Longest first, so "the nearest " goes before "the ".
private val PLACE_LEADS = listOf(
"the nearest ", "the closest ", "a nearby ", "the nearby ", "nearest ", "closest ", "nearby ",
"a ", "an ", "some ",
)
private val PLACE_TAILS = listOf(" near me", " nearby", " near here", " around here", " close by", " closest to me")
private val PLAY_TRIGGERS = listOf("play me ", "play ", "put on ", "listen to ", "i want to hear ", "shuffle ")
private val CALL_TRIGGERS = listOf("call ", "phone ", "dial ", "ring ", "make a call to ", "place a call to ", "start a call with ")
private val OPEN_TRIGGERS = listOf("open ", "launch ", "start ", "go to ", "switch to ", "show me ", "bring up ", "pull up ")
/** Screens "go to X" means rather than a place. */
private val SCREEN_WORDS = setOf("settings", "home", "the home screen", "home screen", "apps", "my apps", "all apps")
/** Padding stripped from the front of a command, longest first where they overlap. */
private val LEAD_FILLERS = listOf(
"hey gearslip", "ok gearslip", "okay gearslip", "gearslip", "hey", "ok", "okay", "um", "uh", "so",
"please", "can you please", "can you", "could you", "would you", "will you", "can i", "could i",
"i want you to", "i want to", "i'd like to", "i would like to", "i wanna", "i need to", "i need you to",
"let's", "lets", "go ahead and", "just", "now", "quickly",
)
/** Longest first, so "right now" goes whole rather than leaving "right". */
private val TAIL_FILLERS = listOf("thank you very much", "thank you", "right now", "for me", "please", "thanks", "now")
private const val APP_CONNECT_MS = 5_000L
private val HELP_WORDS = setOf("help", "commands", "voice commands", "list commands", "show commands")
private val RESUME_WORDS = setOf("music", "the music", "some music", "something", "it", "again", "that", "my music")
private val RESUME_PHRASES = setOf("keep playing", "continue", "keep going", "start the music", "music on", "turn the music back on")
}
@@ -4,7 +4,21 @@ import app.seb3thehacker.gearslip.car.theme.*
import android.content.Intent
import android.net.Uri
import android.provider.Settings
import androidx.compose.animation.core.FastOutSlowInEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.ui.draw.drawBehind
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.scale
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
@@ -30,47 +44,66 @@ import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
/**
* The band that appears while [CarAssistant] is listening, thinking or speaking - a thin strip
* across the top, same idea as [CallOverlay]'s active-call banner, so it never blocks the screen
* the driver was already looking at.
* What the assistant hears and says, laid over the bottom of whatever screen is up: a dark shade
* so white words read over a bright map, and a glow along the bottom edge in the phone's own
* colours that breathes while listening. The shade passes touches through; only the buttons take them.
*/
@Composable
fun AssistantOverlay(modifier: Modifier = Modifier) {
val state by CarAssistant.state.collectAsState()
if (state.phase == CarAssistant.Phase.IDLE) return
val context = LocalContext.current
val listening = state.phase == CarAssistant.Phase.LISTENING && state.ready
val glow = MaterialTheme.colorScheme.primary
val glow2 = MaterialTheme.colorScheme.tertiary
val breathe by rememberInfiniteTransition(label = "glow").animateFloat(
0.55f, 1f, infiniteRepeatable(tween(900, easing = FastOutSlowInEasing), RepeatMode.Reverse), label = "glow",
)
val strength = when (state.phase) {
CarAssistant.Phase.LISTENING -> if (state.ready) breathe else 0.3f
CarAssistant.Phase.ERROR -> 0.3f
else -> 0.6f
}
Surface(
color = MaterialTheme.colorScheme.primaryContainer,
contentColor = MaterialTheme.colorScheme.onPrimaryContainer,
modifier = modifier.fillMaxWidth(),
) {
Row(
Modifier.padding(horizontal = 16.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
PulsingMic(state.phase == CarAssistant.Phase.LISTENING)
Column(Modifier.weight(1f)) {
Text(
when (state.phase) {
CarAssistant.Phase.LISTENING -> "Listening…"
CarAssistant.Phase.THINKING -> "…"
CarAssistant.Phase.SPEAKING -> state.reply
CarAssistant.Phase.ERROR -> state.reply
CarAssistant.Phase.IDLE -> ""
},
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
maxLines = 2,
)
if (state.heard.isNotBlank() && state.phase != CarAssistant.Phase.SPEAKING) {
Text(
"\"${state.heard}\"",
style = MaterialTheme.typography.bodySmall,
Box(
modifier
.fillMaxWidth()
.drawBehind {
drawRect(Brush.verticalGradient(0f to Color.Transparent, 0.45f to Color.Black.copy(alpha = 0.6f), 1f to Color.Black.copy(alpha = 0.85f)))
// A wide, flat oval of light rising from the bottom edge.
scale(1f, 0.22f, Offset(size.width / 2, size.height)) {
drawCircle(
Brush.radialGradient(
0f to glow.copy(alpha = 0.9f * strength), 0.5f to glow2.copy(alpha = 0.35f * strength), 1f to Color.Transparent,
center = Offset(size.width / 2, size.height), radius = size.width * 0.55f,
),
radius = size.width * 0.55f, center = Offset(size.width / 2, size.height),
)
}
}
.padding(start = 16.dp, end = 16.dp, top = 56.dp, bottom = 20.dp),
) {
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(16.dp)) {
PulsingMic(listening)
Column(Modifier.weight(1f), horizontalAlignment = Alignment.CenterHorizontally) {
val (small, big) = when (state.phase) {
CarAssistant.Phase.LISTENING -> "" to state.heard.ifBlank { if (state.ready) "Listening…" else "One moment…" }
CarAssistant.Phase.THINKING -> "" to state.heard.ifBlank { "…" }
else -> state.heard to state.reply
}
if (small.isNotBlank()) {
Text(small, color = Color.White.copy(alpha = 0.7f), style = MaterialTheme.typography.bodyLarge, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
Text(
big,
color = if (big == "Listening…" || big == "One moment…" || big == "…") Color.White.copy(alpha = 0.7f) else Color.White,
style = MaterialTheme.typography.headlineSmall,
fontWeight = FontWeight.SemiBold,
textAlign = TextAlign.Center,
maxLines = 2,
overflow = TextOverflow.Ellipsis,
)
}
if (state.missingPermission) {
GsButton(tone = GsTone.Tonal, onClick = {
context.startActivity(
@@ -0,0 +1,146 @@
package app.seb3thehacker.gearslip.car
import android.Manifest
import android.annotation.SuppressLint
import android.bluetooth.BluetoothA2dp
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothManager
import android.bluetooth.BluetoothProfile
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.PackageManager
import android.os.Handler
import android.os.Looper
import app.seb3thehacker.gearslip.GearslipLog
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* With Bluetooth audio chosen, music only reaches the car if the phone is connected to it over
* Bluetooth. Twenty seconds after the car shows Gearslip, long enough for a car to connect on its
* own, this checks. If the phone isn't connected, the car and the phone's Home screen both say so,
* until it connects or the drive ends.
*
* When the car named its Bluetooth address, only that device counts. Otherwise any audio
* connection does, since there's no telling which device is the car.
*/
object BluetoothAudioCheck {
private val _missing = MutableStateFlow(false)
/** True while Bluetooth audio is chosen and the phone isn't connected to the car for it. */
val missing: StateFlow<Boolean> = _missing
private val _carDismissed = MutableStateFlow(false)
/** The driver tapped OK on the car's card; it stays away for the rest of this drive. */
val carDismissed: StateFlow<Boolean> = _carDismissed
private val log = GearslipLog.tagged("bluetooth")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
private var armed = false
private var receiver: BroadcastReceiver? = null
fun init(context: Context) {
appContext = context.applicationContext
}
/** The car shows Gearslip. Called again whenever the video restarts, so it arms once per drive. */
fun onProjecting() = main.post {
if (armed) return@post
armed = true
main.postDelayed(::start, WAIT_MS)
}
fun onCarDisconnected() = main.post {
armed = false
main.removeCallbacksAndMessages(null)
stopWatching()
_missing.value = false
_carDismissed.value = false
}
fun dismissOnCar() {
_carDismissed.value = true
}
/** "Don't show again" on the car: turns the warning off in Settings, for every car. */
fun neverAgain() {
CarSettings.setBluetoothWarning(false)
log.i("Bluetooth audio warning turned off")
_carDismissed.value = true
_missing.value = false
}
private fun start() {
if (!armed) return
check()
// Connecting later clears the warning, and losing the connection brings it back.
val context = appContext ?: return
val r = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
main.postDelayed(::check, SETTLE_MS)
}
}
context.registerReceiver(
r,
IntentFilter(BluetoothA2dp.ACTION_CONNECTION_STATE_CHANGED).apply {
addAction(BluetoothAdapter.ACTION_STATE_CHANGED)
},
Context.RECEIVER_EXPORTED,
)
receiver = r
}
@SuppressLint("MissingPermission")
private fun check() {
if (!armed) return
val context = appContext ?: return
if (CarSettings.pipeAudio.value) {
set(false, "USB audio is on")
return
}
if (!CarSettings.bluetoothWarning.value) {
set(false, "the warning is turned off")
return
}
if (context.checkSelfPermission(Manifest.permission.BLUETOOTH_CONNECT) != PackageManager.PERMISSION_GRANTED) {
log.i("can't check the audio connection: the Bluetooth permission isn't granted")
return
}
val adapter = context.getSystemService(BluetoothManager::class.java)?.adapter ?: return
if (!adapter.isEnabled) {
set(true, "Bluetooth is off")
return
}
val car = CarBluetooth.carAddress?.uppercase()
adapter.getProfileProxy(context, object : BluetoothProfile.ServiceListener {
override fun onServiceConnected(profile: Int, proxy: BluetoothProfile) {
val connected = runCatching { proxy.connectedDevices }.getOrDefault(emptyList())
adapter.closeProfileProxy(profile, proxy)
val ok = if (car != null) connected.any { it.address.uppercase() == car } else connected.isNotEmpty()
main.post {
set(!ok, if (ok) "connected to the car" else "not connected to the car")
}
}
override fun onServiceDisconnected(profile: Int) {}
}, BluetoothProfile.A2DP)
}
private fun set(missing: Boolean, why: String) {
if (!armed || _missing.value == missing) return
_missing.value = missing
log.i("Bluetooth audio: $why")
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
private const val WAIT_MS = 20_000L
private const val SETTLE_MS = 1_000L
}
@@ -0,0 +1,118 @@
package app.seb3thehacker.gearslip.car
import app.seb3thehacker.gearslip.car.theme.*
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.interaction.MutableInteractionSource
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
/**
* Asks once per car whether to pair it over Bluetooth, then waits while the phone shows its
* pairing dialog. See [CarBluetooth]. With no pairing to do, says when Bluetooth audio isn't
* connected (see [BluetoothAudioCheck]), or else when the phone's location is off (see
* [LocationCheck]). Unlike the call prompt, a tap outside the card does nothing:
* the driver picks Pair or Not now.
*/
@Composable
internal fun BluetoothPairOverlay() {
val prompt by CarBluetooth.prompt.collectAsState()
val bluetoothAudio = !CarSettings.pipeAudio.collectAsState().value
when (prompt) {
CarBluetooth.Prompt.NONE -> {
val missing by BluetoothAudioCheck.missing.collectAsState()
val dismissed by BluetoothAudioCheck.carDismissed.collectAsState()
val warn by CarSettings.bluetoothWarning.collectAsState()
val locationOff by LocationCheck.off.collectAsState()
val locationDismissed by LocationCheck.carDismissed.collectAsState()
if (missing && bluetoothAudio && warn && !dismissed) PairCard(
title = "Bluetooth isn't connected",
body = "Music plays over Bluetooth, but your phone isn't connected to the car. " +
"Connect it from the car's Bluetooth menu or your phone's.",
secondary = "Don't show again" to BluetoothAudioCheck::neverAgain,
primary = "OK" to BluetoothAudioCheck::dismissOnCar,
) else if (locationOff && !locationDismissed) PairCard(
title = "Location is off",
body = "Map apps can't show where you are or give directions. Turn on location " +
"in your phone's quick settings when you're parked.",
secondary = "Don't show again" to LocationCheck::neverAgain,
primary = "OK" to LocationCheck::dismissOnCar,
)
}
CarBluetooth.Prompt.ASK -> PairCard(
title = "Pair with this car?",
body = (if (bluetoothAudio) "Calls and music play" else "Calls play") +
" through the car's speakers once your phone and the car are paired. Your phone " +
"then asks you to confirm a code.",
secondary = "Not now" to CarBluetooth::notNow,
primary = "Pair" to CarBluetooth::pair,
)
CarBluetooth.Prompt.PAIRING -> PairCard(
title = "Confirm on your phone",
body = "Check the code on your phone and tap Pair. The car may ask too.",
secondary = null,
primary = "Hide" to CarBluetooth::hide,
)
}
}
@Composable
private fun PairCard(title: String, body: String, secondary: Pair<String, () -> Unit>?, primary: Pair<String, () -> Unit>) {
Box(
Modifier.fillMaxSize().background(Color.Black.copy(alpha = 0.6f)).clickable(
indication = null,
interactionSource = remember { MutableInteractionSource() },
onClick = {},
),
contentAlignment = Alignment.Center,
) {
Surface(
shape = MaterialTheme.shapes.extraLarge,
color = MaterialTheme.colorScheme.surfaceContainerHigh,
modifier = Modifier.fillMaxWidth(0.85f).padding(16.dp),
) {
Column(
Modifier.padding(32.dp),
verticalArrangement = Arrangement.spacedBy(20.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(title, style = MaterialTheme.typography.headlineSmall, fontWeight = FontWeight.Bold)
Text(
body,
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
)
Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(16.dp)) {
secondary?.let { (label, onClick) ->
GsButton(onClick = onClick, tone = GsTone.Outline, modifier = Modifier.weight(1f).height(56.dp)) {
Text(label, style = MaterialTheme.typography.titleMedium)
}
}
GsButton(onClick = primary.second, modifier = Modifier.weight(1f).height(56.dp)) {
Text(primary.first, style = MaterialTheme.typography.titleMedium)
}
}
}
}
}
}
@@ -22,7 +22,7 @@ class CarApp(
object CarApps {
val all: List<CarApp> = listOf(
CarApp("web", "Web", Icons.Filled.Search) {
CarApp("web", "Browser", Icons.Filled.Search) {
WebApp(content = ProjectedContent.resolve(LocalContext.current))
},
CarApp("phone", "Screen sharing", Icons.Filled.Share) {
@@ -38,6 +38,8 @@ object CarAssistant {
val heard: String = "",
val reply: String = "",
val missingPermission: Boolean = false,
/** False from the press until the microphone is actually open, so the overlay doesn't claim to listen early. */
val ready: Boolean = false,
)
private val _state = MutableStateFlow(State())
@@ -89,6 +91,7 @@ object CarAssistant {
tts?.stop()
_state.value = State(Phase.LISTENING)
listenOnce(
onListening = { _state.value = _state.value.copy(ready = true) },
onPartial = { text -> _state.value = _state.value.copy(heard = text) },
onHeard = { _state.value = _state.value.copy(phase = Phase.THINKING) },
onResult = { text ->
@@ -174,15 +177,16 @@ object CarAssistant {
onSpeaking: () -> Unit = {},
) {
cancel()
listenOnce(onPartial, onHeard = {}, onResult, onError, cue, onSpeaking)
listenOnce(onPartial, onHeard = {}, onResult = onResult, onError = onError, cue = cue, onSpeaking = onSpeaking)
}
/**
* One phrase, through Gearslip's own [SpeechEngine]. The phone's recognizer is only a fallback
* for when the engine itself can't run, since plenty of phones (GrapheneOS among them) have none.
* One phrase, through Gearslip's own [SpeechEngine], or the phone's recognizer when the driver
* chose it in settings. Never the phone's on its own: on most phones that's Google's.
*/
private fun listenOnce(
onPartial: (String) -> Unit,
onListening: () -> Unit = {},
onHeard: () -> Unit,
onResult: (String) -> Unit,
onError: (String) -> Unit,
@@ -190,24 +194,24 @@ object CarAssistant {
onSpeaking: () -> Unit = {},
) {
val context = appContext ?: return onError("Speech isn't ready yet.")
if (CarSettings.systemSpeech.value && SpeechRecognizer.isRecognitionAvailable(context)) {
return systemListen(context, onListening, onPartial, onHeard, onResult, onError)
}
SpeechEngine.listen(
context,
cue = cue,
onSpeaking = onSpeaking,
onPartial = onPartial,
onListening = onListening,
onHeard = onHeard,
onResult = onResult,
onError = { message ->
if (message == SpeechEngine.NOT_LOADED && SpeechRecognizer.isRecognitionAvailable(context)) {
systemListen(context, onPartial, onHeard, onResult, onError)
} else {
onError(message)
}
},
onError = onError,
)
}
private fun systemListen(
context: Context,
onListening: () -> Unit,
onPartial: (String) -> Unit,
onHeard: () -> Unit,
onResult: (String) -> Unit,
@@ -216,7 +220,7 @@ object CarAssistant {
val r = SpeechRecognizer.createSpeechRecognizer(context)
recognizer = r
r.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(params: Bundle?) {}
override fun onReadyForSpeech(params: Bundle?) = onListening()
override fun onBeginningOfSpeech() {}
override fun onRmsChanged(rmsdB: Float) {}
override fun onBufferReceived(buffer: ByteArray?) {}
@@ -0,0 +1,255 @@
package app.seb3thehacker.gearslip.car
import android.Manifest
import android.annotation.SuppressLint
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothDevice
import android.bluetooth.BluetoothManager
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.PackageManager
import android.os.Handler
import android.os.Looper
import androidx.car.app.CarToast
import app.seb3thehacker.gearslip.AppSettings
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
import app.seb3thehacker.gearslip.ServiceDiscovery
import app.seb3thehacker.gearslip.Wire
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import java.security.MessageDigest
/**
* Pairs the phone with the car over Bluetooth the first time they meet, so calls (and music, when
* the driver chose Bluetooth audio) reach the car's speakers. Android Auto does this on its own;
* Gearslip asks first, on the car screen.
*
* The car names its Bluetooth address in its service list. When the driver taps Pair, Gearslip
* opens the car's Bluetooth channel and sends what Android Auto sends:
*
* -> CHANNEL_OPEN_REQUEST <- CHANNEL_OPEN_RESPONSE
* -> PairingRequest { phone_address, method }
* <- PairingResponse { status, already_paired }
*
* then starts pairing from the phone. Two things Android Auto has that Gearslip can't: the
* phone's own Bluetooth address (apps get a placeholder), and confirming the pairing code without
* asking. So a car may turn the request down, and the phone always shows its pairing dialog. The
* channel stays closed unless the driver taps Pair: every new channel is a risk to cars that work.
*/
object CarBluetooth {
enum class Prompt { NONE, ASK, PAIRING }
private val _prompt = MutableStateFlow(Prompt.NONE)
val prompt: StateFlow<Prompt> = _prompt
private val log = GearslipLog.tagged("bluetooth")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
/** This session's car: its channel, and how to open and write to it. */
private class Car(
val service: ServiceDiscovery.BluetoothService,
val open: () -> Unit,
val send: (messageId: Int, body: ByteArray) -> Unit,
)
@Volatile private var car: Car? = null
@Volatile private var bondStarted = false
/** From the Pair tap until it succeeds or fails, whether or not the card still shows. */
@Volatile private var pairing = false
private var receiver: BroadcastReceiver? = null
/** The car's Bluetooth address, when it named one in its service list. */
val carAddress: String? get() = car?.service?.carAddress
/** The car's Bluetooth channel once the driver has asked to pair, -1 otherwise. */
val channelId: Int get() = car?.takeIf { pairing }?.service?.serviceId ?: -1
fun init(context: Context) {
appContext = context.applicationContext
}
/** Called once the car lists its services. Asks on the car screen only if there's pairing to do. */
fun onCarConnected(
service: ServiceDiscovery.BluetoothService,
open: () -> Unit,
send: (messageId: Int, body: ByteArray) -> Unit,
) {
car = Car(service, open, send)
bondStarted = false
log.i("the car offers Bluetooth pairing, methods=${service.pairingMethods}")
val device = device() ?: return
when {
device.bondState == BluetoothDevice.BOND_BONDED -> log.i("already paired with this car")
declined(service.carAddress) -> log.i("not paired, and the driver said not now for this car")
else -> _prompt.value = Prompt.ASK
}
}
fun onCarDisconnected() {
car = null
pairing = false
main.post {
stopWatching()
_prompt.value = Prompt.NONE
}
}
/** The driver tapped Pair. */
fun pair() {
val c = car ?: return
val device = device() ?: return
pairing = true
_prompt.value = Prompt.PAIRING
watchBonding(device)
c.open()
log.i("-> ChannelOpenRequest on channel ${c.service.serviceId}")
// If the car never answers on its channel, pair from the phone anyway.
main.postDelayed({ startBond("the car didn't answer the pairing request") }, CAR_REPLY_WAIT_MS)
main.postDelayed({
if (pairing) finish("Pairing didn't finish. Pair from the car's Bluetooth menu instead.")
}, GIVE_UP_MS)
}
/** The driver tapped Not now: this car isn't asked about again. */
fun notNow() {
car?.service?.carAddress?.let { address ->
val context = appContext ?: return@let
val list = AppSettings.getString(context, KEY_DECLINED, "").split(",").filter { it.isNotEmpty() }
AppSettings.putString(context, KEY_DECLINED, (list + hash(address)).distinct().joinToString(","))
}
_prompt.value = Prompt.NONE
}
/** Hides the pairing card. The phone's own dialog carries on. */
fun hide() {
_prompt.value = Prompt.NONE
}
fun onMessage(messageId: Int, body: ByteArray) {
val c = car ?: return
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
log.i("<- ChannelOpenResponse: status=$status")
if (status != 0) {
main.post { startBond("the car wouldn't open its Bluetooth channel") }
return
}
val method = c.service.pairingMethods.firstOrNull { it == METHOD_NUMERIC_COMPARISON }
?: c.service.pairingMethods.firstOrNull() ?: METHOD_NUMERIC_COMPARISON
c.send(
MSG_PAIRING_REQUEST,
Protobuf.stringField(1, phoneAddress()) + Protobuf.varintField(2, method.toLong()),
)
log.i("-> PairingRequest(method=$method)")
}
MSG_PAIRING_RESPONSE -> {
val fields = Wire.fields(body)
log.i("<- PairingResponse status=${Wire.varint(fields, 1)} already_paired=${Wire.varint(fields, 2) == 1L}")
// Some cars start pairing themselves once they answer; give them a moment first.
main.postDelayed({ startBond("the car answered") }, CAR_BOND_WAIT_MS)
}
// Carries the pairing code. Android shows it to the driver, so it stays out of the log.
MSG_AUTHENTICATION_DATA -> log.i("<- AuthenticationData (${body.size} bytes)")
else -> log.i("<- unhandled message id=$messageId (${body.size} bytes)")
}
}
@SuppressLint("MissingPermission")
private fun startBond(why: String) {
if (bondStarted || !pairing) return
val device = device() ?: return
if (device.bondState != BluetoothDevice.BOND_NONE) return
bondStarted = true
val started = runCatching { device.createBond() }.getOrDefault(false)
log.i("pairing from the phone ($why): ${if (started) "started" else "refused"}")
if (!started) finish("Pairing didn't start. Pair from the car's Bluetooth menu instead.")
}
private fun watchBonding(device: BluetoothDevice) {
val context = appContext ?: return
stopWatching()
val r = object : BroadcastReceiver() {
@Suppress("DEPRECATION")
override fun onReceive(context: Context, intent: Intent) {
val changed = intent.getParcelableExtra<BluetoothDevice>(BluetoothDevice.EXTRA_DEVICE) ?: return
if (changed.address != device.address) return
val state = intent.getIntExtra(BluetoothDevice.EXTRA_BOND_STATE, BluetoothDevice.BOND_NONE)
val previous = intent.getIntExtra(BluetoothDevice.EXTRA_PREVIOUS_BOND_STATE, BluetoothDevice.BOND_NONE)
log.i("bond state $previous -> $state")
when {
state == BluetoothDevice.BOND_BONDING -> bondStarted = true
state == BluetoothDevice.BOND_BONDED -> finish("Paired. Calls now play through the car.")
previous == BluetoothDevice.BOND_BONDING -> finish("Pairing didn't finish. Pair from the car's Bluetooth menu instead.")
}
}
}
context.registerReceiver(r, IntentFilter(BluetoothDevice.ACTION_BOND_STATE_CHANGED), Context.RECEIVER_EXPORTED)
receiver = r
}
private fun finish(message: String) {
pairing = false
stopWatching()
main.removeCallbacksAndMessages(null)
_prompt.value = Prompt.NONE
CarToasts.show(this, message, CarToast.LENGTH_LONG)
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
/** The car as a Bluetooth device, or null (and logged) when Gearslip can't reach Bluetooth. */
private fun device(): BluetoothDevice? {
val context = appContext ?: return null
val address = car?.service?.carAddress ?: return null
if (context.checkSelfPermission(Manifest.permission.BLUETOOTH_CONNECT) != PackageManager.PERMISSION_GRANTED) {
log.i("can't check pairing: the Bluetooth permission isn't granted")
return null
}
val adapter = context.getSystemService(BluetoothManager::class.java)?.adapter
if (adapter == null || !adapter.isEnabled) {
log.i("can't check pairing: Bluetooth is off")
return null
}
if (!BluetoothAdapter.checkBluetoothAddress(address)) {
log.w("the car sent an address that isn't one")
return null
}
return adapter.getRemoteDevice(address)
}
/** Apps get 02:00:00:00:00:00 here, not the real address. Sent anyway, as the field is required. */
@SuppressLint("MissingPermission", "HardwareIds")
private fun phoneAddress(): String =
runCatching { appContext?.getSystemService(BluetoothManager::class.java)?.adapter?.address }.getOrNull()
?: "02:00:00:00:00:00"
private fun declined(address: String): Boolean {
val context = appContext ?: return false
return hash(address) in AppSettings.getString(context, KEY_DECLINED, "").split(",")
}
/** Settings keep a short hash, not the car's address. */
private fun hash(address: String): String =
MessageDigest.getInstance("SHA-256").digest(address.uppercase().toByteArray())
.take(8).joinToString("") { "%02x".format(it) }
private const val KEY_DECLINED = "bluetooth_pair_declined"
private const val CAR_REPLY_WAIT_MS = 5_000L
private const val CAR_BOND_WAIT_MS = 3_000L
private const val GIVE_UP_MS = 90_000L
private const val METHOD_NUMERIC_COMPARISON = 2
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_PAIRING_REQUEST = 32769
private const val MSG_PAIRING_RESPONSE = 32770
private const val MSG_AUTHENTICATION_DATA = 32771
}
@@ -138,18 +138,18 @@ fun CarKeyboard(
}
// Control row: no backspace here - it stays with the letters above, not next to the
// key most likely to be hit right after it by mistake. Dismiss (when offered) is the
// last key, bottom-right corner of the keyboard.
// key most likely to be hit right after it by mistake. Search/enter takes the
// bottom-right corner, where a thumb expects it; dismiss (when offered) sits left of it.
Row(
Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
Key(if (symbols) "abc" else "?123", Modifier.weight(1.4f)) { symbols = !symbols }
Key("space", Modifier.weight(if (onDismiss != null) 3.2f else 4f)) { onTextChange("$text ") }
SubmitKey(Modifier.weight(1.6f), submitIcon, submitImage, onSubmit)
if (onDismiss != null) {
DismissKey(onDismiss, Modifier.weight(1.4f))
}
SubmitKey(Modifier.weight(1.6f), submitIcon, submitImage, onSubmit)
}
}
}
@@ -222,9 +222,9 @@ private fun SubmitKey(modifier: Modifier, search: Boolean, image: ImageVector?,
* Big enough to hit on a bumpy road, small enough that four rows plus a field plus a few results
* still fit. A 480-tall screen sets the floor; a taller one (LIVI's 1280x720) gets bigger keys.
*/
private fun keyHeight(screenHeightDp: Int): Dp = (screenHeightDp * 0.075f).dp.coerceIn(36.dp, 52.dp)
private fun keyHeight(screenHeightDp: Int): Dp = (screenHeightDp * 0.09f).dp.coerceIn(44.dp, 64.dp)
private val LocalKeyHeight = compositionLocalOf { 36.dp }
private val LocalKeyHeight = compositionLocalOf { 44.dp }
private val LETTER_ROWS = listOf(
"qwertyuiop".map { it.toString() },
@@ -27,6 +27,11 @@ object CarKeys {
private val _screenKeys = MutableSharedFlow<ScreenKey>(extraBufferCapacity = 8)
val screenKeys: SharedFlow<ScreenKey> = _screenKeys
private val _focusMoves = MutableSharedFlow<Unit>(extraBufferCapacity = 1)
/** A knob turn or D-pad press, so controls that hide when left alone can come back. */
val focusMoves: SharedFlow<Unit> = _focusMoves
/** Delivers an Android key event to the car display; set while the car UI is showing. */
@Volatile var focusSink: ((keyCode: Int, down: Boolean) -> Unit)? = null
@@ -58,6 +63,7 @@ object CarKeys {
*/
fun onKey(keyCode: Int, down: Boolean, longPress: Boolean): Boolean {
if (keyCode in FOCUS_KEYS) {
_focusMoves.tryEmit(Unit)
focusSink?.invoke(keyCode, down)
return true
}
@@ -86,6 +92,7 @@ object CarKeys {
/** A turn of the rotary knob: each click moves focus one step, like Tab or Shift+Tab. */
fun onRotate(delta: Int) {
val sink = focusSink ?: return
_focusMoves.tryEmit(Unit)
val code = if (delta > 0) KeyEvent.KEYCODE_TAB else KEYCODE_SHIFT_TAB
repeat(kotlin.math.abs(delta).coerceAtMost(MAX_ROTARY_STEPS)) {
sink(code, true)
@@ -95,15 +102,10 @@ object CarKeys {
/**
* The voice key: answers the message on screen if there is one, otherwise starts the
* assistant when it's turned on.
* assistant.
*/
private fun voice() = main.post {
if (VoiceReply.onVoiceKey()) return@post
if (CarSettings.experimentalFeaturesEnabled.value && CarSettings.voiceAssistantEnabled.value) {
CarAssistant.start()
} else {
GearslipLog.i("keys: voice pressed, but the assistant is off in settings")
}
if (!VoiceReply.onVoiceKey()) CarAssistant.start()
}
/** Answers a ringing call; with nothing ringing, opens the phone screen. */
@@ -70,12 +70,8 @@ private class Tile(
val label: String,
val icon: ImageBitmap? = null,
val glyph: ImageVector? = null,
/** Tested from start to finish with Gearslip; drawn with a check mark. */
val verified: Boolean = false,
/** Known not to work; drawn with a red X instead of being left off the launcher. */
val broken: Boolean = false,
/** Starts and can be used, but rough enough not to call working; drawn with a yellow mark. */
val partial: Boolean = false,
val onLongClick: (() -> Unit)? = null,
val onClick: () -> Unit,
)
@@ -187,6 +183,8 @@ private fun groupOf(entry: Entry, mediaPackages: Set<String>): AppGroup {
return when (app.kind) {
"calling" -> AppGroup.PHONE
"navigation", "poi", "parking", "charging" -> AppGroup.MAPS
// Template-only media apps may have no legacy browser/player tile to identify them.
"media" -> AppGroup.MUSIC
"messaging" -> AppGroup.MESSAGING
"settings" -> AppGroup.SETTINGS
else -> AppGroup.OTHER
@@ -241,6 +239,7 @@ internal fun launchEntry(entry: Entry, navigator: CarNavigator, frame: CarEnviro
/** The app launcher: every app as an icon grid, grouped Phone, Maps, Music, Messaging, Web, Screen sharing, Settings. */
@Composable
fun CarLauncher() {
val context = LocalContext.current
val navigator = LocalCarNavigator.current
val frame by CarEnvironment.frame.collectAsState()
@@ -248,18 +247,15 @@ fun CarLauncher() {
val all = rememberAllEntries()
val running = rememberRunningApps()
val showVehicleData = rememberVehicleDataShown()
var showBadgeKey by remember { mutableStateOf(false) }
fun tileOf(entry: Entry): Tile {
// The badges rate an app's own car screens; a messenger shown on Gearslip's own Messages
// screen has none, so it gets no badge.
// Only apps that don't work get a mark. The X rates an app's own car screens; a messenger
// shown on Gearslip's own Messages screen has none, so it never gets one.
val pkg = entry.packageName?.takeIf { entry.messaging == null }
val id = entry.componentId
return Tile(
entry.label, entry.icon, entry.builtIn?.glyph,
verified = pkg != null && !KnownApps.isPartial(pkg) && (KnownApps.works(pkg) || (entry.media != null && KnownApps.playerWorks(pkg))),
broken = pkg != null && (KnownApps.isBroken(pkg) || (entry.template != null && KnownApps.screenBroken(pkg))),
partial = pkg != null && KnownApps.isPartial(pkg),
broken = pkg != null && (KnownApps.isBroken(context, pkg) || (entry.template != null && KnownApps.screenBroken(pkg))),
onLongClick = id?.let { { navigator.showAppMenu(it) } },
) { openApp(entry, id?.let { running[it] }, navigator, frame) }
}
@@ -275,60 +271,6 @@ fun CarLauncher() {
verticalArrangement = Arrangement.spacedBy(12.dp),
) {
items(tiles) { AppTile(it) }
if (tiles.any { it.verified || it.broken || it.partial }) {
// Folded away behind one button: it's reference, read once, not something every
// visit to the launcher needs spelled out under the grid.
item(span = { GridItemSpan(maxLineSpan) }) {
Column(horizontalAlignment = Alignment.CenterHorizontally) {
GsButton(onClick = { showBadgeKey = !showBadgeKey }, tone = GsTone.Neutral) {
Icon(Icons.Filled.Info, contentDescription = null, modifier = Modifier.size(18.dp))
Spacer(Modifier.width(8.dp))
Text(if (showBadgeKey) "Hide badge key" else "What the badges mean")
}
if (showBadgeKey) {
BadgeLegend(
verified = tiles.any { it.verified },
partial = tiles.any { it.partial },
broken = tiles.any { it.broken },
)
}
}
}
}
}
}
/** Says what each badge on a tile means, once, under the grid - only the ones actually in use. */
@Composable
private fun BadgeLegend(verified: Boolean, partial: Boolean, broken: Boolean) {
Column(
Modifier.fillMaxWidth().padding(top = 4.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(4.dp),
) {
if (verified) LegendRow(size = 16.dp, badge = { VerifiedBadge(it) }, text = "Tested and working with Gearslip")
if (partial) LegendRow(size = 16.dp, badge = { PartialBadge(it) }, text = "Runs, but not cleanly, with Gearslip")
if (broken) LegendRow(size = 16.dp, badge = { BrokenBadge(it) }, text = "Doesn't work with Gearslip yet")
}
}
@Composable
private fun LegendRow(size: androidx.compose.ui.unit.Dp, badge: @Composable (androidx.compose.ui.unit.Dp) -> Unit, text: String) {
Row(horizontalArrangement = Arrangement.Center, verticalAlignment = Alignment.CenterVertically) {
badge(size)
Spacer(Modifier.width(8.dp))
Text(text, style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
/** A green disc with a white check: the same mark on every tile and in the legend. */
@Composable
private fun VerifiedBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier = Modifier) {
Box(
modifier.size(size).background(Color(0xFF43A047), CircleShape),
contentAlignment = Alignment.Center,
) {
Icon(Icons.Filled.Check, contentDescription = "Tested and working", tint = Color.White, modifier = Modifier.size(size * 0.66f))
}
}
@@ -343,39 +285,16 @@ private fun BrokenBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier =
}
}
/** A yellow disc with a warning mark: neither the check nor the X, since this app runs but isn't clean about it. */
@Composable
private fun PartialBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier = Modifier) {
Box(
modifier.size(size).background(Color(0xFFF9A825), CircleShape),
contentAlignment = Alignment.Center,
) {
Icon(Icons.Filled.Warning, contentDescription = "Works, but not cleanly, with Gearslip", tint = Color.White, modifier = Modifier.size(size * 0.6f))
}
}
/** Trial: where a tile says whether its app works. See [AppTileBody]. */
private enum class BadgeStyle { ON_ICON, OUTLINE }
private val BADGE_STYLE = BadgeStyle.ON_ICON
@OptIn(ExperimentalFoundationApi::class)
@Composable
private fun AppTile(tile: Tile) {
val status = when {
tile.verified -> Color(0xFF43A047)
tile.partial -> Color(0xFFF9A825)
tile.broken -> Color(0xFFD32F2F)
else -> null
}
val shape = MaterialTheme.shapes.extraLarge
// A long press opens the app's menu (pin, close) - the same gesture everywhere apps are
// shown, here and on the nav bar. GsIconBox marks it in the theme's style.
GsIconBox(
onClick = tile.onClick,
onLongClick = tile.onLongClick,
modifier = Modifier.height(120.dp).then(
if (BADGE_STYLE == BadgeStyle.OUTLINE && status != null) Modifier.border(2.5.dp, status, shape) else Modifier,
),
modifier = Modifier.height(120.dp),
colors = GsColors(MaterialTheme.colorScheme.surfaceVariant, MaterialTheme.colorScheme.onSurfaceVariant),
shape = shape,
) {
@@ -387,16 +306,14 @@ private fun AppTile(tile: Tile) {
) {
Box {
TileIcon(tile)
if (BADGE_STYLE == BadgeStyle.ON_ICON) {
if (tile.broken) {
// Pinned to the icon's lower corner, ringed in the tile's colour so it
// reads as sitting on top of the icon.
val ring = Modifier.align(Alignment.BottomEnd).offset(x = 6.dp, y = 4.dp)
.border(2.dp, MaterialTheme.colorScheme.surfaceVariant, CircleShape)
when {
tile.verified -> VerifiedBadge(22.dp, ring)
tile.broken -> BrokenBadge(22.dp, ring)
tile.partial -> PartialBadge(22.dp, ring)
}
BrokenBadge(
22.dp,
Modifier.align(Alignment.BottomEnd).offset(x = 6.dp, y = 4.dp)
.border(2.dp, MaterialTheme.colorScheme.surfaceVariant, CircleShape),
)
}
}
Spacer(Modifier.height(8.dp))
@@ -33,6 +33,12 @@ object CarServices {
val nav: CarAppConnection by lazy { CarAppConnection(requireNotNull(context)) }
val media: CarMedia by lazy { CarMedia(requireNotNull(context)) }
/** [nav], or null before the car screen has set things up. */
fun navOrNull(): CarAppConnection? = if (context != null) nav else null
/** [media], or null before the car screen has set things up: for code outside the car UI. */
fun mediaOrNull(): CarMedia? = if (context != null) media else null
/**
* A second, independent connection for templated apps that aren't navigation apps - Spotify's
* own Car App Library service is the first of these. Kept apart from [nav] so opening one
@@ -59,6 +65,10 @@ object CarServices {
app.seb3thehacker.gearslip.notify.CarMotion.start(appContext)
app.seb3thehacker.gearslip.call.CarCalls.start(appContext)
CarAssistant.init(appContext)
DirectionsVoice.init(appContext)
CarBluetooth.init(appContext)
BluetoothAudioCheck.init(appContext)
LocationCheck.init(appContext)
CarKeys.init(appContext)
VoiceReply.init(appContext)
// Loaded now, while nothing's waiting on it, so the first reply isn't slower than the rest.
@@ -79,15 +89,22 @@ object CarServices {
if (destination != null && nav.connectedComponent == app.component) {
nav.navigateTo(destination)
} else {
// Another map app taking over ends the old one's route, or it keeps guiding unseen.
if (nav.connectedComponent != app.component) endRoute()
nav.connect(app, frame.width, frame.height, frame.densityDpi, destination)
}
}
fun stopNav() {
navStopped = true
endRoute()
nav.disconnect()
}
private fun endRoute() {
if (nav.navigating) nav.stopNavigating()
}
/** Undoes [stopNav]'s hold so the last map app comes back, same as it would after a restart. */
suspend fun reconnectNav(frame: CarEnvironment.Frame) {
navStopped = false
@@ -121,6 +138,19 @@ object CarServices {
if (CarSettings.pipeAudio.value) CarAudio.request()
}
/**
* Reopens the media app used last, for the car's media button when nothing is open. False
* when there's none on record or it's been uninstalled since.
*/
suspend fun reopenLastMedia(): Boolean {
val ctx = context ?: return false
val last = CarSettings.lastMedia.value?.let(ComponentName::unflattenFromString) ?: return false
val app = withContext(Dispatchers.IO) { MediaCatalog.installed(ctx) }
.firstOrNull { it.component == last } ?: return false
openMedia(app)
return true
}
/** Closes the media app from the car: stops what it's playing and drops the connection. */
fun closeMedia() {
if (media.now.value.playing) media.togglePlay()
@@ -148,7 +178,7 @@ object CarServices {
_hasNavApps.value = navApps.isNotEmpty()
// Never auto-pick one already known to fail outright - a driver who plugs in should
// land on a map, not on the one tile marked with a red X.
val candidates = navApps.filterNot { KnownApps.isBroken(it.component.packageName) }
val candidates = navApps.filterNot { KnownApps.isBroken(ctx, it.component.packageName) }
val last = CarSettings.lastNav.value
// The one used last, or - first run, nothing on record yet - whichever comes first:
// a driver who plugs in for the first time shouldn't have to go pick one by hand
@@ -194,6 +224,32 @@ object CarServices {
return playingPackages.firstNotNullOfOrNull { pkg -> catalog.firstOrNull { it.component.packageName == pkg } }
}
/**
* The car went away mid-song: pause it, as Android Auto does, so it doesn't carry on out of
* the phone's speaker. Unplugging a USB accessory isn't "audio becoming noisy" the way
* headphones are, so the apps won't pause themselves. Every playing session is paused, not
* just the one Gearslip opened: anything the car was playing was going through the car.
*/
fun pausePhoneMedia(why: String = "the car left") {
val ctx = context ?: return
val manager = ctx.getSystemService(MediaSessionManager::class.java)
val listener = ComponentName(ctx, GearslipNotificationListener::class.java)
val playing = try {
manager.getActiveSessions(listener).filter { it.playbackState?.state == PlaybackState.STATE_PLAYING }
} catch (e: SecurityException) {
null
}
if (playing == null) {
// No notification access: the app Gearslip itself is driving is all it can reach.
mediaOrNull()?.let { if (it.now.value.playing) it.togglePlay() }
return
}
playing.forEach { runCatching { it.transportControls.pause() } }
if (playing.isNotEmpty()) {
app.seb3thehacker.gearslip.GearslipLog.i("media: paused ${playing.joinToString { it.packageName }}: $why")
}
}
/** The car went away: nothing should keep running for a screen that no longer exists. */
fun shutdown() {
context?.let { app.seb3thehacker.gearslip.notify.CarMotion.stop(it) }
@@ -202,6 +258,8 @@ object CarServices {
runCatching { media.disconnect() }
runCatching { browse.disconnect() }
CarAssistant.cancel()
// About 900 MB that nothing needs once the car's gone.
app.seb3thehacker.gearslip.speech.SpeechEngine.unload()
CarAudio.release()
_mediaApp.value = null
navStopped = false
@@ -2,6 +2,7 @@ package app.seb3thehacker.gearslip.car
import android.content.Context
import app.seb3thehacker.gearslip.AppSettings
import app.seb3thehacker.gearslip.audio.CarAudio
import app.seb3thehacker.gearslip.car.theme.GsTheme
import app.seb3thehacker.gearslip.car.theme.GsThemes
import kotlinx.coroutines.flow.MutableStateFlow
@@ -27,6 +28,10 @@ object CarSettings {
private const val KEY_OPEN = "car_open_on_connect"
private const val KEY_NIGHT = "car_night_mode"
private const val KEY_PIPE_AUDIO = "car_pipe_audio"
private const val KEY_SPEAK_DIRECTIONS = "car_speak_directions"
private const val KEY_BLUETOOTH_WARNING = "car_bluetooth_warning"
private const val KEY_LOCATION_WARNING = "car_location_warning"
private const val KEY_AUDIO_CHOICE_ASKED = "car_audio_choice_asked"
private const val KEY_LAST_NAV = "car_last_nav"
private const val KEY_LAST_MEDIA = "car_last_media"
private const val KEY_AUTOPLAY = "car_autoplay"
@@ -36,9 +41,8 @@ object CarSettings {
private const val KEY_PINNED = "car_pinned_apps"
private const val KEY_APP_ORDER = "car_app_order"
private const val KEY_EXPERIMENTAL_FEATURES = "car_experimental_features"
private const val KEY_VOICE_ASSISTANT = "car_voice_assistant"
private const val KEY_SYSTEM_SPEECH = "car_system_speech"
private const val KEY_CAR_SENSORS = "car_sensors_enabled"
private const val KEY_MEDIA_SEARCH = "car_media_search_enabled"
private const val KEY_SEEN_SAFETY_WARNING = "car_seen_safety_warning"
private const val KEY_UNITS = "car_units"
private const val KEY_WHATS_NEW = "car_whats_new_after_updates"
@@ -57,6 +61,9 @@ object CarSettings {
private val openFlow = MutableStateFlow(HOME)
private val nightFlow = MutableStateFlow(NightMode.AUTO)
private val pipeAudioFlow = MutableStateFlow(true)
private val speakDirectionsFlow = MutableStateFlow(true)
private val bluetoothWarningFlow = MutableStateFlow(true)
private val locationWarningFlow = MutableStateFlow(true)
private val lastNavFlow = MutableStateFlow<String?>(null)
private val lastMediaFlow = MutableStateFlow<String?>(null)
private val autoplayFlow = MutableStateFlow(false)
@@ -70,9 +77,8 @@ object CarSettings {
private val pinnedFlow = MutableStateFlow<Set<String>>(emptySet())
private val appOrderFlow = MutableStateFlow<List<String>>(emptyList())
private val experimentalFeaturesFlow = MutableStateFlow(false)
private val voiceAssistantFlow = MutableStateFlow(false)
private val systemSpeechFlow = MutableStateFlow(false)
private val carSensorsFlow = MutableStateFlow(false)
private val mediaSearchFlow = MutableStateFlow(false)
private val seenSafetyWarningFlow = MutableStateFlow(false)
private val unitsFlow = MutableStateFlow(Units.AUTO)
@@ -148,22 +154,41 @@ object CarSettings {
* Whether a media app's sound is captured and sent over the car link. On by default, since
* sound in the car is the point of the exercise; the platform allows the capture only behind a
* consent dialog, which [app.seb3thehacker.gearslip.audio.CarAudio] raises once the car has
* somewhere to play it. Turned off, the app plays through the phone's normal output instead.
* somewhere to play it. Turned off ("Bluetooth" in the settings), the app plays through the
* phone's normal output instead, and Android never asks.
*/
val pipeAudio: StateFlow<Boolean> = pipeAudioFlow
/** Gearslip reads the map app's turns aloud on the car's directions channel; see DirectionsVoice. */
val speakDirections: StateFlow<Boolean> = speakDirectionsFlow
/** With Bluetooth audio, warn when the phone isn't connected to the car; see BluetoothAudioCheck. */
val bluetoothWarning: StateFlow<Boolean> = bluetoothWarningFlow
/** Warn on the car when the phone's location is off; see LocationCheck. */
val locationWarning: StateFlow<Boolean> = locationWarningFlow
/**
* Whether the driver has already been offered Bluetooth after turning down the capture
* dialog. Offered once only: someone who chose to keep the USB route shouldn't be nagged.
*/
/** False until the driver picks USB or Bluetooth, in setup or in settings. */
val pipeAudioChosen: Boolean
get() = AppSettings.getString(app, KEY_PIPE_AUDIO, "") != ""
val audioChoiceAsked: Boolean
get() = AppSettings.getString(app, KEY_AUDIO_CHOICE_ASKED, "false") == "true"
fun markAudioChoiceAsked() = AppSettings.putString(app, KEY_AUDIO_CHOICE_ASKED, "true")
/**
* Off by default: a driver has to opt into experimental features as a whole before any of
* them - the voice assistant included - show up to turn on individually.
*/
val experimentalFeaturesEnabled: StateFlow<Boolean> = experimentalFeaturesFlow
/**
* The voice assistant is experimental and off by default; a driver has to turn it on
* knowingly, from the settings screen, after reading the warning shown there. This flag
* alone does not run the assistant - [experimentalFeaturesEnabled] must also be on.
*/
val voiceAssistantEnabled: StateFlow<Boolean> = voiceAssistantFlow
/** Listen with the phone's own speech service instead of Gearslip's on-device model. */
val systemSpeech: StateFlow<Boolean> = systemSpeechFlow
/**
* Vehicle data is experimental and off by default: the launcher's tile for it, and the
@@ -173,13 +198,6 @@ object CarSettings {
*/
val carSensorsEnabled: StateFlow<Boolean> = carSensorsFlow
/**
* Searching inside a media app is experimental and off by default: many apps claim to take
* searches and then ignore them or play something else. The media screen's search button
* shows only once this and [experimentalFeaturesEnabled] are both on.
*/
val mediaSearchEnabled: StateFlow<Boolean> = mediaSearchFlow
/**
* Whether the car screen's own "drive safely" warning has already been shown and dismissed
* once, from this same install. Separate from the phone side's own compat/safety dialogs -
@@ -206,6 +224,11 @@ object CarSettings {
autoplayFlow.value = AppSettings.getString(app, KEY_AUTOPLAY, "false") == "true"
playerTitleFlow.value = AppSettings.getString(app, KEY_PLAYER_TITLE, "false") == "true"
pipeAudioFlow.value = AppSettings.getString(app, KEY_PIPE_AUDIO, "true") == "true"
// Until the driver sets it, on for USB audio and off for Bluetooth, where the map app's
// own voice already reaches the car.
speakDirectionsFlow.value = AppSettings.getString(app, KEY_SPEAK_DIRECTIONS, pipeAudioFlow.value.toString()) == "true"
bluetoothWarningFlow.value = AppSettings.getString(app, KEY_BLUETOOTH_WARNING, "true") == "true"
locationWarningFlow.value = AppSettings.getString(app, KEY_LOCATION_WARNING, "true") == "true"
nightFlow.value = runCatching {
NightMode.valueOf(AppSettings.getString(app, KEY_NIGHT, NightMode.AUTO.name))
}.getOrDefault(NightMode.AUTO)
@@ -214,9 +237,8 @@ object CarSettings {
appOrderFlow.value = AppSettings.getString(app, KEY_APP_ORDER, "")
.split(",").filter { it.isNotEmpty() }
experimentalFeaturesFlow.value = AppSettings.getString(app, KEY_EXPERIMENTAL_FEATURES, "false") == "true"
voiceAssistantFlow.value = AppSettings.getString(app, KEY_VOICE_ASSISTANT, "false") == "true"
systemSpeechFlow.value = AppSettings.getString(app, KEY_SYSTEM_SPEECH, "false") == "true"
carSensorsFlow.value = AppSettings.getString(app, KEY_CAR_SENSORS, "false") == "true"
mediaSearchFlow.value = AppSettings.getString(app, KEY_MEDIA_SEARCH, "false") == "true"
seenSafetyWarningFlow.value = AppSettings.getString(app, KEY_SEEN_SAFETY_WARNING, "false") == "true"
unitsFlow.value = runCatching {
Units.valueOf(AppSettings.getString(app, KEY_UNITS, Units.AUTO.name))
@@ -265,9 +287,32 @@ object CarSettings {
AppSettings.putString(app, KEY_AUTOPLAY, on.toString())
}
/**
* Takes effect at once: switching to Bluetooth mid-drive stops the capture, and switching
* back asks for it again (or waits for the car, if none is connected).
*/
fun setPipeAudio(on: Boolean) {
pipeAudioFlow.value = on
// Saved even when unchanged, so picking the default still counts as a choice.
AppSettings.putString(app, KEY_PIPE_AUDIO, on.toString())
if (pipeAudioFlow.value == on) return
pipeAudioFlow.value = on
if (AppSettings.getString(app, KEY_SPEAK_DIRECTIONS, "") == "") speakDirectionsFlow.value = on
if (on) CarAudio.request() else CarAudio.release()
}
fun setSpeakDirections(on: Boolean) {
speakDirectionsFlow.value = on
AppSettings.putString(app, KEY_SPEAK_DIRECTIONS, on.toString())
}
fun setBluetoothWarning(on: Boolean) {
bluetoothWarningFlow.value = on
AppSettings.putString(app, KEY_BLUETOOTH_WARNING, on.toString())
}
fun setLocationWarning(on: Boolean) {
locationWarningFlow.value = on
AppSettings.putString(app, KEY_LOCATION_WARNING, on.toString())
}
fun setExperimentalFeaturesEnabled(on: Boolean) {
@@ -275,9 +320,9 @@ object CarSettings {
AppSettings.putString(app, KEY_EXPERIMENTAL_FEATURES, on.toString())
}
fun setVoiceAssistantEnabled(on: Boolean) {
voiceAssistantFlow.value = on
AppSettings.putString(app, KEY_VOICE_ASSISTANT, on.toString())
fun setSystemSpeech(on: Boolean) {
systemSpeechFlow.value = on
AppSettings.putString(app, KEY_SYSTEM_SPEECH, on.toString())
}
fun setCarSensorsEnabled(on: Boolean) {
@@ -285,11 +330,6 @@ object CarSettings {
AppSettings.putString(app, KEY_CAR_SENSORS, on.toString())
}
fun setMediaSearchEnabled(on: Boolean) {
mediaSearchFlow.value = on
AppSettings.putString(app, KEY_MEDIA_SEARCH, on.toString())
}
fun setNightMode(mode: NightMode) {
nightFlow.value = mode
AppSettings.putString(app, KEY_NIGHT, mode.name)
@@ -53,14 +53,16 @@ fun CarSettingsScreen() {
val units by CarSettings.units.collectAsState()
val whatsNew by CarSettings.whatsNewAfterUpdates.collectAsState()
val pipeAudio by CarSettings.pipeAudio.collectAsState()
val speakDirections by CarSettings.speakDirections.collectAsState()
val bluetoothWarning by CarSettings.bluetoothWarning.collectAsState()
val locationWarning by CarSettings.locationWarning.collectAsState()
val autoplay by CarSettings.autoplay.collectAsState()
val playerTitle by CarSettings.playerTitle.collectAsState()
val theme by CarSettings.appTheme.collectAsState()
val uiTheme by CarSettings.uiTheme.collectAsState()
val experimentalFeatures by CarSettings.experimentalFeaturesEnabled.collectAsState()
val voiceAssistant by CarSettings.voiceAssistantEnabled.collectAsState()
val carSensors by CarSettings.carSensorsEnabled.collectAsState()
val mediaSearch by CarSettings.mediaSearchEnabled.collectAsState()
val systemSpeech by CarSettings.systemSpeech.collectAsState()
val display by CarEnvironment.display.collectAsState()
val vehicle by CarEnvironment.vehicle.collectAsState()
val context = LocalContext.current
@@ -126,17 +128,49 @@ fun CarSettingsScreen() {
) { CarSettings.setAutoplay(it) }
ChoiceRow(
"Media audio",
listOf(false to "Phone audio", true to "Car audio"),
"Car audio",
listOf(false to "Bluetooth", true to "USB"),
pipeAudio,
) { CarSettings.setPipeAudio(it) }
Text(
"Car audio sends the phone's sound to the car over USB. Android asks for permission " +
"when the car connects, and a few apps block it. Phone audio plays as the phone " +
"normally would, for example over Bluetooth to the car.",
"USB sends the phone's sound through Gearslip. Android asks to share your screen each " +
"drive, and a few apps block it. Bluetooth plays as the phone normally would, and " +
"Android never asks.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
if (!pipeAudio) {
ChoiceRow(
"Warn when Bluetooth isn't connected",
listOf(false to "Off", true to "On"),
bluetoothWarning,
) { CarSettings.setBluetoothWarning(it) }
}
ChoiceRow(
"Warn when location is off",
listOf(false to "Off", true to "On"),
locationWarning,
) { CarSettings.setLocationWarning(it) }
ChoiceRow(
"Spoken directions",
listOf(false to "Off", true to "On"),
speakDirections,
) { CarSettings.setSpeakDirections(it) }
Text(
"Gearslip reads each turn aloud through the car's speakers, and the music dips while it " +
"speaks. Turn it off if your map app's own voice already plays in the car.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
ExperimentalFeatureRow(
title = "Use the phone's speech recognition",
description = "Listens through the phone's own speech service instead of Gearslip's. On " +
"most phones that's Google's, which may send your voice to Google. GrapheneOS has " +
"none, so Gearslip keeps using its own.",
checked = systemSpeech,
) { CarSettings.setSystemSpeech(it) }
WebPageSetting()
@@ -148,7 +182,7 @@ fun CarSettingsScreen() {
experimentalFeatures,
) { CarSettings.setExperimentalFeaturesEnabled(it) }
if (experimentalFeatures) {
ExperimentalFeaturesList(voiceAssistant, carSensors, mediaSearch)
ExperimentalFeaturesList(carSensors)
}
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -205,21 +239,11 @@ private fun <T> ChoiceRow(title: String, options: List<Pair<T, String>>, selecte
* so the list reads fine whether there's one entry or several.
*/
@Composable
private fun ExperimentalFeaturesList(voiceAssistant: Boolean, carSensors: Boolean, mediaSearch: Boolean) {
private fun ExperimentalFeaturesList(carSensors: Boolean) {
Column(
Modifier.fillMaxWidth(),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
ExperimentalFeatureRow(
title = "Voice assistant",
description = "Push-to-talk and the car's voice button start listening: navigate, " +
"play media, call, or open an app by saying so. Can mishear you or act on the " +
"wrong command - keep your eyes on the road, and confirm anything it does " +
"before you rely on it.",
warning = true,
checked = voiceAssistant,
) { CarSettings.setVoiceAssistantEnabled(it) }
ExperimentalFeatureRow(
title = "Vehicle data",
description = "Adds a Vehicle data tile: a live, raw dump of the head unit's sensor " +
@@ -228,13 +252,6 @@ private fun ExperimentalFeaturesList(voiceAssistant: Boolean, carSensors: Boolea
"guess, not a fact, until you've watched one change for yourself.",
checked = carSensors,
) { CarSettings.setCarSensorsEnabled(it) }
ExperimentalFeatureRow(
title = "Media search",
description = "Adds a search button to the media screen. Some apps ignore the search " +
"or play something else.",
checked = mediaSearch,
) { CarSettings.setMediaSearchEnabled(it) }
}
}
@@ -272,7 +289,7 @@ private fun ExperimentalFeatureRow(
}
}
/** The URL (or raw HTML) the Web app shows. Edited with [CarKeyboard] - no phone in hand needed. */
/** The URL (or raw HTML) the Browser app shows. Edited with [CarKeyboard] - no phone in hand needed. */
@Composable
private fun WebPageSetting() {
val context = LocalContext.current
@@ -297,7 +314,7 @@ private fun WebPageSetting() {
)
}
Text(
"Shown by the Web app on the launcher. Applies on the next connection.",
"Shown by the Browser app on the launcher. Applies on the next connection.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
@@ -0,0 +1,62 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.platform.LocalAccessibilityManager
import androidx.compose.ui.semantics.LiveRegionMode
import androidx.compose.ui.semantics.liveRegion
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.withTimeoutOrNull
/** Draw inside the encoded car display; a normal Android Toast would appear on the phone. */
@Composable
internal fun CarToastOverlay(modifier: Modifier = Modifier) {
val current by CarToasts.message.collectAsState()
val message = current ?: return
val accessibility = LocalAccessibilityManager.current
LaunchedEffect(message, accessibility) {
val timeout = accessibility?.calculateRecommendedTimeoutMillis(
message.durationMs, containsIcons = false, containsText = true, containsControls = false,
) ?: message.durationMs
// The full time, unless another toast is waiting: then it gives way after the minimum.
delay(minOf(timeout, CarToasts.MIN_SHOWN_MS))
withTimeoutOrNull((timeout - CarToasts.MIN_SHOWN_MS).coerceAtLeast(0)) {
CarToasts.waiting.first { it > 0 }
}
CarToasts.dismiss(message)
}
// A non-interactive overlay leaves the map's touch handling and focus intact.
Box(
modifier
.padding(horizontal = 24.dp, vertical = 16.dp)
.widthIn(max = 560.dp)
.clip(RoundedCornerShape(20.dp))
.background(MaterialTheme.colorScheme.surfaceContainerHigh)
.semantics { liveRegion = LiveRegionMode.Polite },
) {
Text(
message.text,
modifier = Modifier.padding(horizontal = 20.dp, vertical = 12.dp),
color = MaterialTheme.colorScheme.onSurface,
style = MaterialTheme.typography.titleMedium,
maxLines = 3,
overflow = TextOverflow.Ellipsis,
)
}
}
@@ -0,0 +1,65 @@
package app.seb3thehacker.gearslip.car
import androidx.annotation.MainThread
import androidx.car.app.CarToast
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* One toast at a time across navigation and browse apps. A newer one waits its turn instead of
* replacing the one on screen, which gets at least [MIN_SHOWN_MS] before it gives way.
*/
@MainThread
internal object CarToasts {
// Identity matters: repeating identical text must restart its display timer.
class Message(val owner: Any, val text: String, val durationMs: Long)
/** Long enough to read a short toast at a glance; a waiting toast cuts the rest short. */
const val MIN_SHOWN_MS = 2_000L
/** A burst beyond this many is stale by the time it would show; the oldest waiting go first. */
private const val MAX_WAITING = 3
private val current = MutableStateFlow<Message?>(null)
val message: StateFlow<Message?> = current.asStateFlow()
private val queue = ArrayDeque<Message>()
private val _waiting = MutableStateFlow(0)
val waiting: StateFlow<Int> = _waiting.asStateFlow()
fun show(owner: Any, text: String, duration: Int) {
if (text.isBlank()) return
// Longer than a phone toast: the driver only glances at the screen now and then.
val durationMs = if (duration == CarToast.LENGTH_LONG) 7_000L else 4_000L
val next = Message(owner, text, durationMs)
val shown = current.value
when {
shown == null -> current.value = next
// The same words again restart the one on screen rather than queueing a copy.
shown.text == text -> current.value = next
queue.lastOrNull()?.text == text -> Unit
else -> {
queue.addLast(next)
while (queue.size > MAX_WAITING) queue.removeFirst()
}
}
_waiting.value = queue.size
}
fun dismiss(message: Message) {
// An old timer must not dismiss a newer toast, even if its text is identical.
if (current.value === message) advance()
}
fun clear(owner: Any) {
// Disconnecting the map must not clear a toast belonging to the browse app.
queue.removeAll { it.owner === owner }
if (current.value?.owner === owner) advance() else _waiting.value = queue.size
}
private fun advance() {
current.value = queue.removeFirstOrNull()
_waiting.value = queue.size
}
}
@@ -2,6 +2,7 @@ package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.Image
import androidx.compose.foundation.clickable
import androidx.compose.foundation.LocalIndication
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.BoxWithConstraints
@@ -43,6 +44,10 @@ import androidx.compose.material3.dynamicDarkColorScheme
import androidx.compose.material3.dynamicLightColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.hapticfeedback.HapticFeedback
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.platform.LocalView
import androidx.compose.runtime.State
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
@@ -120,12 +125,16 @@ fun CarUi() {
controlsWake.tick()
}
}
// The knob counts as a touch: the map's buttons can't be found while they're faded out.
LaunchedEffect(controlsWake) { CarKeys.focusMoves.collect { controlsWake.touch() } }
remember(appContext) { Prefetch.warm(appContext, force = true); CarServices.init(appContext) }
DisposableEffect(Unit) { onDispose { CarServices.shutdown() } }
// Ask for the phone's sound as soon as the car connects, not only when a media app opens:
// a video in the web browser or a map's voice needs it too. Asks once, after the car offers
// somewhere to play it.
LaunchedEffect(Unit) { if (CarSettings.pipeAudio.value) app.seb3thehacker.gearslip.audio.CarAudio.request() }
// A new version gets one card in the top right, a few seconds in and only while parked.
LaunchedEffect(Unit) { app.seb3thehacker.gearslip.update.UpdateChecker.onCarConnected(appContext) }
// The map follows the light outside, whatever the app's own theme is set to.
LaunchedEffect(darkOutside) { CarServices.nav.pushConfiguration() }
@@ -150,14 +159,27 @@ fun CarUi() {
VoiceReply.state.value.phase != VoiceReply.Phase.IDLE -> VoiceReply.cancel()
else -> navigator.back()
}
CarKeys.ScreenKey.HOME, CarKeys.ScreenKey.MAP -> navigator.home()
CarKeys.ScreenKey.MEDIA -> if (CarServices.mediaApp.value != null) navigator.media() else navigator.apps()
// Home is the dashboard, over the map, so Back from it lands on the map.
CarKeys.ScreenKey.HOME -> { navigator.home(); navigator.dashboard() }
CarKeys.ScreenKey.MAP -> navigator.home()
CarKeys.ScreenKey.MEDIA ->
if (CarServices.mediaApp.value != null || CarServices.reopenLastMedia()) navigator.media()
else navigator.apps()
CarKeys.ScreenKey.PHONE -> BuiltInApps.Phone.open(navigator)
}
}
}
// The car screen is drawn on the phone, so every long press or text-handle drag would buzz
// the phone in its mount. Nobody feels that while driving; it's only noise.
val view = LocalView.current
DisposableEffect(view) {
view.isHapticFeedbackEnabled = false
onDispose { }
}
CompositionLocalProvider(
LocalHapticFeedback provides NoHaptics,
LocalDensity provides Density(base.density * scale, base.fontScale),
LocalCarNavigator provides navigator,
LocalDarkOutside provides darkOutside,
@@ -165,6 +187,7 @@ fun CarUi() {
LocalGsTheme provides GsThemes.byId(uiThemeId),
) {
MaterialTheme(colorScheme = colors) {
CompositionLocalProvider(LocalIndication provides FocusRingIndication(MaterialTheme.colorScheme.primary)) {
Surface(
Modifier
.fillMaxSize()
@@ -224,6 +247,9 @@ fun CarUi() {
}
}
}
CarToastOverlay(Modifier.align(Alignment.BottomCenter))
CallPromptOverlay()
BluetoothPairOverlay()
PopupTimer()
// The dashboard is already listing them, so it needs no popup.
NotificationPopup(
@@ -231,14 +257,15 @@ fun CarUi() {
modifier = Modifier.align(Alignment.TopEnd),
)
// Above the popup: a call is more urgent than any notification.
VoiceHelpOverlay(Modifier.align(Alignment.Center))
CallOverlay(Modifier.align(Alignment.TopCenter))
AssistantOverlay(Modifier.align(Alignment.TopCenter))
AssistantOverlay(Modifier.align(Alignment.BottomCenter))
if (app.seb3thehacker.gearslip.BuildConfig.DEBUG) {
// Bottom corner, not top: a screen's own header almost always puts
// something in the top-right (settings, search, a Liked Songs
// shortcut) and this badge used to sit right on top of it, hiding
// whatever the screen drew there.
DevBuildBadge(Modifier.align(Alignment.BottomEnd))
DebugBuildBadge(Modifier.align(Alignment.BottomEnd))
}
}
// The keyboard takes the nav bar's row instead of squeezing in above it -
@@ -252,6 +279,7 @@ fun CarUi() {
if (safetyWarningSeen) CarWhatsNew()
}
}
}
}
}
}
@@ -354,7 +382,7 @@ private fun AppWontStart(app: String, onBack: () -> Unit) {
* that matters while driving, since a debug build is slower and less tested than what ships.
*/
@Composable
private fun DevBuildBadge(modifier: Modifier = Modifier) {
private fun DebugBuildBadge(modifier: Modifier = Modifier) {
Surface(
modifier = modifier.padding(6.dp),
color = MaterialTheme.colorScheme.error,
@@ -362,7 +390,7 @@ private fun DevBuildBadge(modifier: Modifier = Modifier) {
shape = MaterialTheme.shapes.small,
) {
Text(
"DEV BUILD",
"DEBUG BUILD",
style = MaterialTheme.typography.labelSmall,
fontWeight = FontWeight.Bold,
modifier = Modifier.padding(horizontal = 8.dp, vertical = 3.dp),
@@ -443,12 +471,8 @@ private fun CarNavBar(navigator: CarNavigator) {
NavHomeButton(navigator, entries, running)
Spacer(Modifier.width(6.dp))
NavItem(MediaIcons.Apps, navigator.current == CarScreen.Apps) { navigator.apps() }
val experimentalFeatures by CarSettings.experimentalFeaturesEnabled.collectAsState()
val voiceEnabled by CarSettings.voiceAssistantEnabled.collectAsState()
if (experimentalFeatures && voiceEnabled) {
Spacer(Modifier.width(6.dp))
NavAssistantIcon()
}
Spacer(Modifier.width(6.dp))
NavAssistantIcon()
// Pinned shortcuts, then whatever else is open right now. A pinned app that's also
// open stays in its pinned spot, marked with a dot, instead of showing up twice.
@@ -879,3 +903,8 @@ private fun ChargingGlyph(modifier: Modifier = Modifier, tint: Color = MaterialT
drawPath(bolt, tint)
}
}
/** Stands in for the phone's vibration motor on the car screen: does nothing. */
private object NoHaptics : HapticFeedback {
override fun performHapticFeedback(hapticFeedbackType: HapticFeedbackType) {}
}
@@ -0,0 +1,199 @@
package app.seb3thehacker.gearslip.car
import androidx.car.app.model.CarText
import androidx.car.app.model.Distance
import androidx.car.app.navigation.model.Maneuver
import androidx.car.app.navigation.model.NavigationTemplate
import androidx.car.app.navigation.model.RoutingInfo
import androidx.car.app.navigation.model.Step
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
/**
* The next turn on the car's instrument cluster, over its NavigationStatusService (Service
* field 8), the way Android Auto feeds it:
*
* -> NavigationStatus { status = 1 (1 active, 2 inactive, 3 rerouting) }
* -> NavigationNextTurnEvent { road = 1; turn_side = 2; event = 3; turn_number = 5; turn_angle = 6 }
* -> NavigationNextTurnDistanceEvent { distance_meters = 1; time_to_turn_seconds = 2;
* display_distance_e3 = 3; display_distance_unit = 4 }
*
* Newer cars also take a NavigationState message; the deprecated pair above is what a 2018
* Uconnect understands. The turn comes from the map app's trip (what the Car App Library hands
* to clusters), or from its routing card when it sends no trip.
*/
class ClusterNav(
val channelId: Int,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
) {
@Volatile private var running = false
private var sentStatus = -1
private var sentTurn: String? = null
private var sentDistance: String? = null
fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
GearslipLog.i("cluster: <- ChannelOpenResponse: status=$status")
if (status == 0) start()
}
// The car saying when it wants turns. Sending all along is harmless, so these are only logged.
MSG_START -> GearslipLog.i("cluster: <- start")
MSG_STOP -> GearslipLog.i("cluster: <- stop")
else -> GearslipLog.i("cluster: <- unhandled message id=$messageId (${body.size} bytes)")
}
}
private fun start() {
if (running) return
running = true
Thread({
while (running) {
runCatching { tick() }.onFailure { GearslipLog.w("cluster: ${it.message}") }
Thread.sleep(TICK_MS)
}
}, "gearslip-cluster").start()
}
fun stop() {
running = false
}
private fun tick() {
val turn = NavTurn.current()
val status = when {
turn == null -> STATUS_INACTIVE
turn.rerouting -> STATUS_REROUTING
else -> STATUS_ACTIVE
}
if (status != sentStatus) {
sendOnChannel(MSG_STATUS, Protobuf.varintField(1, status.toLong()))
GearslipLog.i("cluster: -> ${STATUS_NAMES[status]}")
sentStatus = status
if (status == STATUS_INACTIVE) { sentTurn = null; sentDistance = null }
}
if (turn == null || status == STATUS_REROUTING) return
sendTurn(turn.step)
turn.distance?.let { sendDistance(it, turn.seconds) }
}
private fun sendTurn(step: Step) {
val maneuver = step.maneuver
val (event, side) = maneuver?.let { eventFor(it.type) } ?: (EVENT_UNKNOWN to SIDE_UNSPECIFIED)
val road = step.road.text().ifBlank { step.cue.text() }
val key = "$road|$event|$side|${maneuver?.roundaboutExitNumber}"
if (key == sentTurn) return
sentTurn = key
var body = Protobuf.stringField(1, road) +
Protobuf.varintField(2, side.toLong()) +
Protobuf.varintField(3, event.toLong())
maneuver?.roundaboutExitNumber?.takeIf { it > 0 }?.let { body += Protobuf.varintField(5, it.toLong()) }
maneuver?.roundaboutExitAngle?.takeIf { it > 0 }?.let { body += Protobuf.varintField(6, it.toLong()) }
sendOnChannel(MSG_TURN, body)
GearslipLog.i("cluster: -> turn event=$event side=$side road=\"$road\"")
}
/**
* The Car App Library's distance units use the same numbers as the protocol's (1 metres ...
* 7 yards), so the app's own display choice passes straight through.
*/
private fun sendDistance(distance: Distance, seconds: Long?) {
val meters = NavTurn.toMeters(distance)
val key = "${distance.displayDistance}|${distance.displayUnit}|${seconds ?: -1}"
if (key == sentDistance) return
sentDistance = key
sendOnChannel(
MSG_DISTANCE,
Protobuf.varintField(1, meters.toLong()) +
Protobuf.varintField(2, (seconds ?: 0L).coerceAtLeast(0)) +
Protobuf.varintField(3, (distance.displayDistance * 1000).toLong()) +
Protobuf.varintField(4, distance.displayUnit.toLong()),
)
}
private fun CarText?.text() = this?.toString().orEmpty()
companion object {
private const val TICK_MS = 500L
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_START = 32769
private const val MSG_STOP = 32770
private const val MSG_STATUS = 32771
private const val MSG_TURN = 32772
private const val MSG_DISTANCE = 32773
private const val STATUS_ACTIVE = 1
private const val STATUS_INACTIVE = 2
private const val STATUS_REROUTING = 3
private val STATUS_NAMES = mapOf(1 to "navigating", 2 to "not navigating", 3 to "rerouting")
private const val SIDE_LEFT = 1
private const val SIDE_RIGHT = 2
private const val SIDE_UNSPECIFIED = 3
private const val EVENT_UNKNOWN = 0
/** The car's turn icon and side for a Car App Library manoeuvre. */
fun eventFor(type: Int): Pair<Int, Int> = when (type) {
Maneuver.TYPE_DEPART -> 1 to SIDE_UNSPECIFIED
Maneuver.TYPE_NAME_CHANGE -> 2 to SIDE_UNSPECIFIED
Maneuver.TYPE_KEEP_LEFT, Maneuver.TYPE_TURN_SLIGHT_LEFT -> 3 to SIDE_LEFT
Maneuver.TYPE_KEEP_RIGHT, Maneuver.TYPE_TURN_SLIGHT_RIGHT -> 3 to SIDE_RIGHT
Maneuver.TYPE_TURN_NORMAL_LEFT -> 4 to SIDE_LEFT
Maneuver.TYPE_TURN_NORMAL_RIGHT -> 4 to SIDE_RIGHT
Maneuver.TYPE_TURN_SHARP_LEFT -> 5 to SIDE_LEFT
Maneuver.TYPE_TURN_SHARP_RIGHT -> 5 to SIDE_RIGHT
Maneuver.TYPE_U_TURN_LEFT -> 6 to SIDE_LEFT
Maneuver.TYPE_U_TURN_RIGHT -> 6 to SIDE_RIGHT
Maneuver.TYPE_ON_RAMP_SLIGHT_LEFT, Maneuver.TYPE_ON_RAMP_NORMAL_LEFT, Maneuver.TYPE_ON_RAMP_SHARP_LEFT,
Maneuver.TYPE_ON_RAMP_U_TURN_LEFT -> 7 to SIDE_LEFT
Maneuver.TYPE_ON_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_ON_RAMP_NORMAL_RIGHT, Maneuver.TYPE_ON_RAMP_SHARP_RIGHT,
Maneuver.TYPE_ON_RAMP_U_TURN_RIGHT -> 7 to SIDE_RIGHT
Maneuver.TYPE_OFF_RAMP_SLIGHT_LEFT, Maneuver.TYPE_OFF_RAMP_NORMAL_LEFT -> 8 to SIDE_LEFT
Maneuver.TYPE_OFF_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_OFF_RAMP_NORMAL_RIGHT -> 8 to SIDE_RIGHT
Maneuver.TYPE_FORK_LEFT -> 9 to SIDE_LEFT
Maneuver.TYPE_FORK_RIGHT -> 9 to SIDE_RIGHT
Maneuver.TYPE_MERGE_LEFT -> 10 to SIDE_LEFT
Maneuver.TYPE_MERGE_RIGHT -> 10 to SIDE_RIGHT
Maneuver.TYPE_MERGE_SIDE_UNSPECIFIED -> 10 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_ENTER_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_CCW -> 11 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_EXIT_CCW -> 12 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW_WITH_ANGLE,
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW_WITH_ANGLE -> 13 to SIDE_UNSPECIFIED
Maneuver.TYPE_STRAIGHT -> 14 to SIDE_UNSPECIFIED
Maneuver.TYPE_FERRY_BOAT, Maneuver.TYPE_FERRY_BOAT_LEFT, Maneuver.TYPE_FERRY_BOAT_RIGHT -> 16 to SIDE_UNSPECIFIED
Maneuver.TYPE_FERRY_TRAIN, Maneuver.TYPE_FERRY_TRAIN_LEFT, Maneuver.TYPE_FERRY_TRAIN_RIGHT -> 17 to SIDE_UNSPECIFIED
Maneuver.TYPE_DESTINATION, Maneuver.TYPE_DESTINATION_STRAIGHT -> 19 to SIDE_UNSPECIFIED
Maneuver.TYPE_DESTINATION_LEFT -> 19 to SIDE_LEFT
Maneuver.TYPE_DESTINATION_RIGHT -> 19 to SIDE_RIGHT
else -> EVENT_UNKNOWN to SIDE_UNSPECIFIED
}
}
}
/** One turn as the car wants it, from whichever source the map app fills. */
internal class NavTurn(val step: Step, val distance: Distance?, val seconds: Long?, val rerouting: Boolean) {
companion object {
/** The map app's next turn: from its trip (what clusters get), or its routing card. */
fun current(): NavTurn? {
val nav = CarServices.navOrNull() ?: return null
nav.trip.value?.let { trip ->
val step = trip.steps.firstOrNull() ?: return@let
val estimate = trip.stepTravelEstimates.firstOrNull()
return NavTurn(step, estimate?.remainingDistance, estimate?.remainingTimeSeconds?.takeIf { it >= 0 }, trip.isLoading)
}
val info = (nav.template.value as? NavigationTemplate)?.navigationInfo as? RoutingInfo ?: return null
val step = info.currentStep ?: return null
return NavTurn(step, info.currentDistance, null, info.isLoading)
}
fun toMeters(d: Distance): Int = (d.displayDistance * when (d.displayUnit) {
Distance.UNIT_KILOMETERS, Distance.UNIT_KILOMETERS_P1 -> 1000.0
Distance.UNIT_MILES, Distance.UNIT_MILES_P1 -> 1609.344
Distance.UNIT_FEET -> 0.3048
Distance.UNIT_YARDS -> 0.9144
else -> 1.0
}).toInt()
}
}
@@ -0,0 +1,326 @@
package app.seb3thehacker.gearslip.car
import android.content.Context
import android.os.Bundle
import android.speech.tts.TextToSpeech
import android.speech.tts.UtteranceProgressListener
import androidx.car.app.model.CarText
import androidx.car.app.model.Distance
import androidx.car.app.navigation.model.Maneuver
import androidx.car.app.navigation.model.Step
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.audio.AudioLink
import java.io.File
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.CountDownLatch
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
import kotlin.math.roundToInt
/**
* Spoken turn-by-turn on the car's directions channel (the GUIDANCE audio sink).
*
* A map app on the phone speaks its own prompts as navigation audio, which Android can't hand to
* Gearslip without root, and which the phone mutes while USB audio runs. So Gearslip reads the
* same turn the instrument cluster gets ([NavTurn]) and says it itself: once when the turn comes
* up, again a kilometre or so out on a long stretch, and once more just before it. The words are
* rendered with the phone's text-to-speech to a file, never played on the phone, and streamed to
* the car, which ducks the music under them the way it does for real Android Auto.
*
* Nothing spoken is logged: road names are where the driver is going.
*/
object DirectionsVoice {
private enum class Stage { FAR, PREPARED, DONE }
private var appContext: Context? = null
private var tts: TextToSpeech? = null
@Volatile private var ttsReady = false
private val pending = ConcurrentHashMap<String, CountDownLatch>()
@Volatile private var link: AudioLink? = null
@Volatile private var running = false
private val speaker = Executors.newSingleThreadExecutor { Thread(it, "gearslip-directions").apply { isDaemon = true } }
@Volatile private var latest = 0
private var turnKey: String? = null
private var stage = Stage.DONE
fun init(context: Context) {
if (tts != null) return
appContext = context.applicationContext
tts = TextToSpeech(appContext) { status -> ttsReady = status == TextToSpeech.SUCCESS }.apply {
setOnUtteranceProgressListener(object : UtteranceProgressListener() {
override fun onStart(utteranceId: String?) {}
override fun onDone(utteranceId: String?) { pending.remove(utteranceId)?.countDown() }
@Deprecated("Deprecated in Java, no replacement on this API level")
override fun onError(utteranceId: String?) { pending.remove(utteranceId)?.countDown() }
})
}
}
/** The car opened its directions sink: start following the map app's turns. */
fun attach(link: AudioLink) {
this.link = link
GearslipLog.i("guidance: the car offered a directions sink at ${link.sampleRate}Hz x${link.channels}")
if (running) return
running = true
turnKey = null
Thread({
while (running) {
runCatching { tick() }.onFailure { GearslipLog.w("guidance: ${it.message}") }
Thread.sleep(TICK_MS)
}
}, "gearslip-directions-watch").apply { isDaemon = true }.start()
}
fun detach(link: AudioLink) {
if (this.link !== link) return
this.link = null
running = false
}
private fun tick() {
if (!CarSettings.speakDirections.value) return
val turn = NavTurn.current()
if (turn == null) {
turnKey = null
return
}
if (turn.rerouting) return
val step = turn.step
val instruction = instruction(step) ?: return
val meters = turn.distance?.let(NavTurn::toMeters)
val close = (turn.seconds != null && turn.seconds <= NEAR_SECONDS) || (meters != null && meters <= NEAR_METERS)
val key = "${step.road.text()}|${step.maneuver?.type}|${step.maneuver?.roundaboutExitNumber}|${step.cue.text()}"
if (key != turnKey) {
turnKey = key
if (step.maneuver?.type == Maneuver.TYPE_DEPART) {
stage = Stage.DONE
return
}
if (close || turn.distance == null) {
say(instruction)
stage = Stage.DONE
} else {
say("In ${spoken(turn.distance)}, $instruction")
stage = if (meters != null && meters > PREPARE_FROM_METERS) Stage.FAR else Stage.PREPARED
}
return
}
when {
stage != Stage.DONE && close -> {
say(instruction)
stage = Stage.DONE
}
stage == Stage.FAR && meters != null && meters <= PREPARE_AT_METERS -> {
say("In ${spoken(turn.distance)}, $instruction")
stage = Stage.PREPARED
}
}
}
/** What to do, in words: built from the manoeuvre when the app names one, else the app's own text. */
private fun instruction(step: Step): String? {
val road = step.road.text()
val onto = if (road.isBlank()) "" else " onto $road"
val toward = if (road.isBlank()) "" else " toward $road"
val maneuver = step.maneuver
val built = when (maneuver?.type) {
Maneuver.TYPE_TURN_SLIGHT_LEFT -> "Turn slightly left$onto"
Maneuver.TYPE_TURN_SLIGHT_RIGHT -> "Turn slightly right$onto"
Maneuver.TYPE_TURN_NORMAL_LEFT -> "Turn left$onto"
Maneuver.TYPE_TURN_NORMAL_RIGHT -> "Turn right$onto"
Maneuver.TYPE_TURN_SHARP_LEFT -> "Turn sharply left$onto"
Maneuver.TYPE_TURN_SHARP_RIGHT -> "Turn sharply right$onto"
Maneuver.TYPE_U_TURN_LEFT, Maneuver.TYPE_U_TURN_RIGHT -> "Make a U-turn"
Maneuver.TYPE_KEEP_LEFT -> "Keep left$toward"
Maneuver.TYPE_KEEP_RIGHT -> "Keep right$toward"
Maneuver.TYPE_FORK_LEFT -> "Keep left at the fork$toward"
Maneuver.TYPE_FORK_RIGHT -> "Keep right at the fork$toward"
Maneuver.TYPE_ON_RAMP_SLIGHT_LEFT, Maneuver.TYPE_ON_RAMP_NORMAL_LEFT, Maneuver.TYPE_ON_RAMP_SHARP_LEFT,
Maneuver.TYPE_ON_RAMP_U_TURN_LEFT -> "Take the ramp on the left$onto"
Maneuver.TYPE_ON_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_ON_RAMP_NORMAL_RIGHT, Maneuver.TYPE_ON_RAMP_SHARP_RIGHT,
Maneuver.TYPE_ON_RAMP_U_TURN_RIGHT -> "Take the ramp on the right$onto"
Maneuver.TYPE_OFF_RAMP_SLIGHT_LEFT, Maneuver.TYPE_OFF_RAMP_NORMAL_LEFT -> "Take the exit on the left$toward"
Maneuver.TYPE_OFF_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_OFF_RAMP_NORMAL_RIGHT -> "Take the exit on the right$toward"
Maneuver.TYPE_MERGE_LEFT -> "Merge left$onto"
Maneuver.TYPE_MERGE_RIGHT -> "Merge right$onto"
Maneuver.TYPE_MERGE_SIDE_UNSPECIFIED -> "Merge$onto"
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW_WITH_ANGLE,
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW_WITH_ANGLE ->
maneuver.roundaboutExitNumber.takeIf { it in 1..ORDINALS.size }
?.let { "At the roundabout, take the ${ORDINALS[it - 1]} exit$onto" }
?: "Go through the roundabout$onto"
Maneuver.TYPE_ROUNDABOUT_ENTER_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_CCW -> "Enter the roundabout"
Maneuver.TYPE_ROUNDABOUT_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_EXIT_CCW -> "Exit the roundabout$onto"
Maneuver.TYPE_STRAIGHT -> "Continue straight$onto"
Maneuver.TYPE_NAME_CHANGE -> "Continue$onto"
Maneuver.TYPE_FERRY_BOAT, Maneuver.TYPE_FERRY_BOAT_LEFT, Maneuver.TYPE_FERRY_BOAT_RIGHT -> "Take the ferry"
Maneuver.TYPE_FERRY_TRAIN, Maneuver.TYPE_FERRY_TRAIN_LEFT, Maneuver.TYPE_FERRY_TRAIN_RIGHT -> "Take the train"
Maneuver.TYPE_DESTINATION, Maneuver.TYPE_DESTINATION_STRAIGHT -> "Your destination is ahead"
Maneuver.TYPE_DESTINATION_LEFT -> "Your destination is on the left"
Maneuver.TYPE_DESTINATION_RIGHT -> "Your destination is on the right"
else -> null
}
return built ?: step.cue.text().ifBlank { null }
}
/** The app's own rounded distance, read the way a person would say it. */
private fun spoken(d: Distance): String {
val n = d.displayDistance
val number = if (n % 1.0 == 0.0) n.toInt().toString() else "%.1f".format(n)
val one = n == 1.0
return when (d.displayUnit) {
Distance.UNIT_KILOMETERS, Distance.UNIT_KILOMETERS_P1 -> if (one) "1 kilometer" else "$number kilometers"
Distance.UNIT_MILES, Distance.UNIT_MILES_P1 -> when (n) {
0.25 -> "a quarter mile"
0.5 -> "half a mile"
0.75 -> "three quarters of a mile"
1.0 -> "1 mile"
else -> "$number miles"
}
Distance.UNIT_FEET -> "$number feet"
Distance.UNIT_YARDS -> "$number yards"
else -> "${n.roundToInt()} meters"
}
}
/** Newest wins: a prompt still waiting when the next one arrives is already out of date. */
private fun say(text: String) {
val id = ++latest
speaker.execute {
if (id != latest) return@execute
runCatching { play(text, id) }.onFailure { GearslipLog.w("guidance: could not speak a turn: ${it.message}") }
}
}
private fun play(text: String, id: Int) {
val engine = tts ?: return
val context = appContext ?: return
val sink = link ?: return
if (!ttsReady) {
GearslipLog.w("guidance: text-to-speech isn't ready, a turn went unspoken")
return
}
val file = File(context.cacheDir, "directions.wav")
val utterance = "gearslip-directions-$id"
val done = CountDownLatch(1)
pending[utterance] = done
if (engine.synthesizeToFile(text, Bundle(), file, utterance) != TextToSpeech.SUCCESS) {
pending.remove(utterance)
return
}
if (!done.await(SYNTH_TIMEOUT_S, TimeUnit.SECONDS)) {
pending.remove(utterance)
GearslipLog.w("guidance: text-to-speech took too long, a turn went unspoken")
return
}
val pcm = Wav.read(file)?.let { Wav.convert(it, sink.sampleRate, sink.channels) } ?: run {
GearslipLog.w("guidance: text-to-speech gave audio Gearslip can't read")
return
}
stream(sink, pcm)
GearslipLog.i("guidance: spoke a turn (${pcm.size / (2 * sink.channels) * 1000 / sink.sampleRate} ms)")
}
/** Start, the words in 20 ms chunks at about real time, a little silence, Stop. */
private fun stream(sink: AudioLink, pcm: ByteArray) {
val frame = 2 * sink.channels
val chunk = sink.sampleRate / 50 * frame
val chunkMs = 20L
val padded = pcm + ByteArray(sink.sampleRate / 5 * frame) // 200 ms tail so the last word isn't clipped
sink.start()
try {
val began = System.currentTimeMillis()
var offset = 0
var sent = 0
while (offset < padded.size && link === sink) {
val size = minOf(chunk, padded.size - offset)
sink.write(padded.copyOfRange(offset, offset + size), size)
offset += size
sent++
val ahead = sent * chunkMs - (System.currentTimeMillis() - began)
if (ahead > LEAD_MS) Thread.sleep(ahead - LEAD_MS)
}
// Let the car play what it has before closing the stream on it.
Thread.sleep(LEAD_MS)
} finally {
sink.stop()
}
}
private fun CarText?.text() = this?.toString().orEmpty()
/** 16-bit PCM from the WAV text-to-speech writes, reshaped for the car's sink. */
private object Wav {
class Pcm(val samples: ShortArray, val rate: Int, val channels: Int)
fun read(file: File): Pcm? {
val bytes = file.readBytes()
if (bytes.size < 12 || String(bytes, 0, 4) != "RIFF" || String(bytes, 8, 4) != "WAVE") return null
val buf = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN)
var pos = 12
var rate = 0
var channels = 0
var bits = 0
while (pos + 8 <= bytes.size) {
val name = String(bytes, pos, 4)
val size = buf.getInt(pos + 4)
val body = pos + 8
// Some engines write a streaming WAV with a zero or oversized data length.
val end = if (size <= 0 || body + size > bytes.size) bytes.size else body + size
when (name) {
"fmt " -> {
channels = buf.getShort(body + 2).toInt()
rate = buf.getInt(body + 4)
bits = buf.getShort(body + 14).toInt()
}
"data" -> {
if (bits != 16 || rate <= 0 || channels <= 0) return null
val samples = ShortArray((end - body) / 2)
buf.position(body)
buf.asShortBuffer().get(samples, 0, samples.size)
return Pcm(samples, rate, channels)
}
}
pos = end + (end and 1)
}
return null
}
/** Mixes to mono, resamples linearly, then copies to each of the car's channels. */
fun convert(pcm: Pcm, rate: Int, channels: Int): ByteArray {
val frames = pcm.samples.size / pcm.channels
val mono = FloatArray(frames) { f ->
var sum = 0f
for (c in 0 until pcm.channels) sum += pcm.samples[f * pcm.channels + c]
sum / pcm.channels
}
val outFrames = (frames.toLong() * rate / pcm.rate).toInt()
val out = ByteBuffer.allocate(outFrames * channels * 2).order(ByteOrder.LITTLE_ENDIAN)
val step = pcm.rate.toDouble() / rate
for (i in 0 until outFrames) {
val at = i * step
val j = at.toInt().coerceAtMost(frames - 1)
val next = (j + 1).coerceAtMost(frames - 1)
val v = mono[j] + (mono[next] - mono[j]) * (at - j).toFloat()
val s = v.roundToInt().coerceIn(Short.MIN_VALUE.toInt(), Short.MAX_VALUE.toInt()).toShort()
repeat(channels) { out.putShort(s) }
}
return out.array()
}
}
private const val TICK_MS = 500L
private const val NEAR_SECONDS = 12L
private const val NEAR_METERS = 120
private const val PREPARE_FROM_METERS = 2_500
private const val PREPARE_AT_METERS = 1_000
private const val LEAD_MS = 100L
private const val SYNTH_TIMEOUT_S = 10L
private val ORDINALS = listOf("first", "second", "third", "fourth", "fifth", "sixth", "seventh", "eighth")
}
@@ -0,0 +1,84 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.IndicationNodeFactory
import androidx.compose.foundation.interaction.FocusInteraction
import androidx.compose.foundation.interaction.InteractionSource
import androidx.compose.material3.ripple
import androidx.compose.ui.geometry.CornerRadius
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.ContentDrawScope
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.node.DelegatableNode
import androidx.compose.ui.node.DelegatingNode
import androidx.compose.ui.node.DrawModifierNode
import androidx.compose.ui.node.invalidateDraw
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.launch
/**
* What every car button shows: the usual ripple when pressed, and a ring in the accent color
* while the knob or D-pad is on it. Cars without a knob never see the ring, since nothing takes
* focus while the screen is in touch mode.
*
* It replaces the theme's ripple (LocalIndication) for the whole car UI, so any `clickable` that
* doesn't pass its own indication gets the ring for free.
*/
internal class FocusRingIndication(private val color: Color) : IndicationNodeFactory {
private val ripple = ripple()
override fun create(interactionSource: InteractionSource): DelegatableNode =
FocusRingNode(ripple.create(interactionSource), interactionSource, color)
override fun equals(other: Any?) = other is FocusRingIndication && other.color == color
override fun hashCode() = color.hashCode()
}
private class FocusRingNode(
rippleNode: DelegatableNode,
private val source: InteractionSource,
private val color: Color,
) : DelegatingNode(), DrawModifierNode {
private var focused = false
init { delegate(rippleNode) }
override fun onAttach() {
coroutineScope.launch {
var holding = 0
source.interactions.collect {
when (it) {
is FocusInteraction.Focus -> holding++
is FocusInteraction.Unfocus -> holding--
}
val now = holding > 0
if (now != focused) {
focused = now
invalidateDraw()
}
}
}
}
override fun ContentDrawScope.draw() {
drawContent()
if (!focused) return
// Inside the button's bounds, since most buttons clip to their shape: drawn outside, a
// rounded tile would cut its own ring off.
val width = RING_WIDTH.toPx()
val radius = minOf(CORNER.toPx(), size.minDimension / 2)
drawRoundRect(
color,
topLeft = Offset(width / 2, width / 2),
size = Size(size.width - width, size.height - width),
cornerRadius = CornerRadius(radius),
style = Stroke(width),
)
}
private companion object {
val RING_WIDTH = 3.dp
val CORNER = 16.dp
}
}
@@ -0,0 +1,93 @@
package app.seb3thehacker.gearslip.car
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.location.LocationManager
import android.os.Handler
import android.os.Looper
import app.seb3thehacker.gearslip.GearslipLog
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* Map apps can't find the car with the phone's location turned off. Ten seconds after the car
* shows Gearslip, this checks, and the car says so until location comes on or the drive ends.
*/
object LocationCheck {
private val _off = MutableStateFlow(false)
/** True while the phone's location is turned off during a drive. */
val off: StateFlow<Boolean> = _off
private val _carDismissed = MutableStateFlow(false)
/** The driver tapped OK on the car's card; it stays away for the rest of this drive. */
val carDismissed: StateFlow<Boolean> = _carDismissed
private val log = GearslipLog.tagged("location")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
private var armed = false
private var receiver: BroadcastReceiver? = null
fun init(context: Context) {
appContext = context.applicationContext
}
/** The car shows Gearslip. Called again whenever the video restarts, so it arms once per drive. */
fun onProjecting() = main.post {
if (armed) return@post
armed = true
main.postDelayed(::start, WAIT_MS)
}
fun onCarDisconnected() = main.post {
armed = false
main.removeCallbacksAndMessages(null)
stopWatching()
_off.value = false
_carDismissed.value = false
}
fun dismissOnCar() {
_carDismissed.value = true
}
/** "Don't show again" on the car: turns the warning off in Settings, for every car. */
fun neverAgain() {
CarSettings.setLocationWarning(false)
log.i("location warning turned off")
_carDismissed.value = true
_off.value = false
}
private fun start() {
if (!armed) return
check()
// Turning location on clears the warning, and turning it off again brings it back.
val context = appContext ?: return
val r = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) = check()
}
context.registerReceiver(r, IntentFilter(LocationManager.MODE_CHANGED_ACTION), Context.RECEIVER_EXPORTED)
receiver = r
}
private fun check() {
if (!armed) return
val manager = appContext?.getSystemService(LocationManager::class.java) ?: return
val off = CarSettings.locationWarning.value && !manager.isLocationEnabled
if (_off.value == off) return
_off.value = off
log.i(if (off) "the phone's location is off" else "the phone's location is on")
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
private const val WAIT_MS = 10_000L
}
@@ -61,7 +61,6 @@ import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import app.seb3thehacker.gearslip.audio.CarAudio
import app.seb3thehacker.gearslip.media.CarMedia
import app.seb3thehacker.gearslip.media.MediaApp
import app.seb3thehacker.gearslip.media.MediaArt
@@ -88,11 +87,9 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
val queue by media.queue.collectAsStateWithLifecycle()
val browse by media.browse.collectAsStateWithLifecycle()
val rejection by media.rejection.collectAsStateWithLifecycle()
val capture by CarAudio.capture.collectAsStateWithLifecycle()
val search by media.search.collectAsStateWithLifecycle()
var tab by remember { mutableStateOf(Tab.NOW_PLAYING) }
var searching by remember { mutableStateOf(false) }
val searchAllowed = CarSettings.experimentalFeaturesEnabled.collectAsState().value && CarSettings.mediaSearchEnabled.collectAsState().value
// The connection outlives this screen: the home screen's player is the same one.
LaunchedEffect(app) { CarServices.openMedia(app) }
@@ -174,7 +171,6 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
} else {
CompactPlayer(now, media, Modifier.weight(1f).fillMaxWidth())
}
AudioBadge(capture)
}
Column(Modifier.weight(1.2f).fillMaxHeight()) {
Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -182,7 +178,7 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
MediaTab("Browse", tab == Tab.BROWSE, Modifier.weight(1f)) { tab = Tab.BROWSE }
MediaTab("Up next", tab == Tab.QUEUE, Modifier.weight(1f)) { tab = Tab.QUEUE }
MediaTab("Lyrics", tab == Tab.LYRICS, Modifier.weight(1f)) { tab = Tab.LYRICS }
if (searchAllowed && (now.canSearch || media.canSearchLibrary)) {
if (now.canSearch || media.canSearchLibrary) {
GsIconButton(
Icons.Filled.Search,
"Search ${app.label}",
@@ -447,17 +443,6 @@ private sealed interface LyricsState {
class Found(val result: LyricsResult) : LyricsState
}
@Composable
private fun AudioBadge(capture: CarAudio.Capture) {
val text = when (capture) {
CarAudio.Capture.ON -> "Audio to car"
CarAudio.Capture.ASKING -> "Allow audio capture on the phone"
CarAudio.Capture.DENIED -> "Audio stays on the phone"
CarAudio.Capture.OFF -> return
}
Text(text, style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
@Composable
private fun Notice(title: String, detail: String?, action: Pair<String, () -> Unit>? = null) {
Column(
@@ -72,7 +72,7 @@ internal class BuiltInApp(
)
internal object BuiltInApps {
val Web = BuiltInApp("web", "Web", Icons.Filled.Search, CarScreen.App("web")) { it.open("web") }
val Web = BuiltInApp("web", "Browser", Icons.Filled.Search, CarScreen.App("web")) { it.open("web") }
val ScreenSharing = BuiltInApp("phone", "Screen sharing", Icons.Filled.Share, CarScreen.App("phone")) { it.open("phone") }
val Phone = BuiltInApp("dialer", "Phone", Icons.Filled.Call, CarScreen.App("dialer")) { it.open("dialer") }
val VehicleData = BuiltInApp("vehicle", "Vehicle data", Icons.Filled.Info, CarScreen.VehicleData) { it.vehicleData() }
@@ -32,6 +32,7 @@ import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.produceState
@@ -89,6 +90,31 @@ internal fun placeCall(context: Context, number: String) {
}
}
/** A call another car screen wants to place - a number on a place page - waiting on the driver. */
internal object CallPrompt {
val pending = kotlinx.coroutines.flow.MutableStateFlow<Contact?>(null)
fun ask(name: String, number: String) {
pending.value = Contact(name, number)
}
}
/** The dialer's own confirmation, drawn over the whole car screen for [CallPrompt]. */
@Composable
internal fun CallPromptOverlay() {
val context = LocalContext.current
val contact by CallPrompt.pending.collectAsState()
val c = contact ?: return
ConfirmCallOverlay(
c,
onConfirm = {
CallPrompt.pending.value = null
placeCall(context, c.number)
},
onDismiss = { CallPrompt.pending.value = null },
)
}
private const val TAB_CONTACTS = 0
private const val TAB_KEYPAD = 1
@@ -138,29 +138,47 @@ internal fun PlayModeRow(
val repeatOn = repeatKnown && modes.repeat != PlaybackStateCompat.REPEAT_MODE_NONE
val icon = size * 0.52f
Row(modifier, horizontalArrangement = Arrangement.SpaceEvenly, verticalAlignment = Alignment.CenterVertically) {
GsIconButton(
if (liked == true) Icons.Filled.Favorite else Icons.Filled.FavoriteBorder,
if (liked == true) "Unlike" else "Like",
media::toggleLike,
enabled = liked != null,
tone = if (liked == true) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
GsIconButton(
MediaIcons.Shuffle, if (shuffleOn) "Shuffle off" else "Shuffle on",
media::toggleShuffle,
enabled = shuffleKnown || now.shuffleAction != null,
tone = if (shuffleOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
GsIconButton(
if (modes.repeat == PlaybackStateCompat.REPEAT_MODE_ONE) MediaIcons.RepeatOne else MediaIcons.Repeat,
"Repeat",
media::cycleRepeat,
enabled = repeatKnown || now.repeatAction != null,
tone = if (repeatOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
// An app's own like, shuffle or repeat button shows whether it's on only through its icon,
// so that icon is drawn, as Android Auto does, and lit when its name says it's on.
// Gearslip's own stand in for apps that report a heart rating or shuffle and repeat modes.
val likeAction = now.likeAction
if (likeAction != null) {
CustomActionButton(likeAction, media, size, toggle = true)
} else {
GsIconButton(
if (liked == true) Icons.Filled.Favorite else Icons.Filled.FavoriteBorder,
if (liked == true) "Unlike" else "Like",
media::toggleLike,
enabled = liked != null,
tone = if (liked == true) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
}
val shuffleAction = now.shuffleAction
if (shuffleAction != null) {
CustomActionButton(shuffleAction, media, size, toggle = true)
} else {
GsIconButton(
MediaIcons.Shuffle, if (shuffleOn) "Shuffle off" else "Shuffle on",
media::toggleShuffle,
enabled = shuffleKnown,
tone = if (shuffleOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
}
val repeatAction = now.repeatAction
if (repeatAction != null) {
CustomActionButton(repeatAction, media, size, toggle = true)
} else {
GsIconButton(
if (modes.repeat == PlaybackStateCompat.REPEAT_MODE_ONE) MediaIcons.RepeatOne else MediaIcons.Repeat,
"Repeat",
media::cycleRepeat,
enabled = repeatKnown,
tone = if (repeatOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
}
now.otherActions.take(maxOther).forEach { CustomActionButton(it, media, size) }
trailing()
}
@@ -171,12 +189,12 @@ internal fun PlayModeRow(
* not Gearslip's, so it's loaded from there. Its name stands in if the icon won't load.
*/
@Composable
private fun CustomActionButton(action: CustomAction, media: CarMedia, size: Dp) {
private fun CustomActionButton(action: CustomAction, media: CarMedia, size: Dp, toggle: Boolean = false) {
val icon = rememberActionIcon(action)
GsIconBox(
onClick = { media.custom(action) },
modifier = Modifier.size(size),
colors = gsColors(GsTone.Neutral),
colors = gsColors(if (toggle && action.on) GsTone.Primary else GsTone.Neutral),
) {
if (icon != null) {
Icon(icon, action.name, Modifier.size(size * 0.52f))
@@ -45,6 +45,7 @@ import androidx.car.app.model.MessageTemplate
import androidx.car.app.model.PaneTemplate
import androidx.car.app.model.Row as CarRow
import androidx.car.app.model.SearchTemplate
import androidx.compose.foundation.clickable
import androidx.car.app.model.Section
import androidx.car.app.model.SectionedItemTemplate
import androidx.car.app.model.TabTemplate
@@ -238,6 +239,9 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
// initialSearchText would fight whatever is being typed.
var query by remember { mutableStateOf(template.initialSearchText.orEmpty()) }
val delegate = template.searchCallbackDelegate
// An app that opens on results ("navigate to Starbucks") asks for the keyboard to stay down,
// so the results aren't hidden under it. Tapping the field brings it up.
var typing by remember { mutableStateOf(template.isShowKeyboardByDefault) }
ContentSurface(modifier) {
// Back button and field share a row: on a 480px screen the keyboard and the results
@@ -251,7 +255,7 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
SearchField(
query,
template.searchHint.orEmpty(),
Modifier.weight(1f),
Modifier.weight(1f).clickable { typing = true },
onClear = {
query = ""
delegate.textChanged("")
@@ -265,14 +269,17 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
else ItemListColumn(template.itemList, modifier = Modifier.fillMaxSize())
}
CarKeyboard(
text = query,
onTextChange = {
query = it
delegate.textChanged(it)
},
onSubmit = { delegate.submitted(query) },
)
if (typing) {
CarKeyboard(
text = query,
onTextChange = {
query = it
delegate.textChanged(it)
},
onSubmit = { delegate.submitted(query) },
onDismiss = { typing = false },
)
}
}
}
@@ -3,8 +3,10 @@ package app.seb3thehacker.gearslip.car
import app.seb3thehacker.gearslip.car.theme.*
import androidx.compose.runtime.produceState
import androidx.compose.runtime.remember
import androidx.compose.ui.focus.onFocusChanged
import androidx.compose.material3.LocalContentColor
import androidx.compose.ui.graphics.ColorFilter
import androidx.compose.ui.graphics.luminance
import androidx.compose.ui.graphics.ImageBitmap
import androidx.compose.foundation.Image
import kotlinx.coroutines.Dispatchers
@@ -29,6 +31,7 @@ import androidx.compose.foundation.lazy.grid.items
import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Call
import androidx.compose.material.icons.filled.ArrowBack
import androidx.compose.material.icons.filled.Menu
import androidx.compose.material.icons.filled.Warning
@@ -231,7 +234,9 @@ internal fun RightEdgeControls(
// the first touch on one wakes them rather than pressing it.
val awake = LocalMapControlsWake.current.awake
val pan by CarServices.nav.panMode.collectAsState()
val visible = awake || pan
// Kept up while the knob is on one of them, or the ring would fade out along with the button.
var holdsFocus by remember { mutableStateOf(false) }
val visible = awake || pan || holdsFocus
val alpha by androidx.compose.animation.core.animateFloatAsState(
targetValue = if (visible) 1f else 0f,
animationSpec = androidx.compose.animation.core.tween(if (visible) 150 else 500),
@@ -240,6 +245,7 @@ internal fun RightEdgeControls(
Column(
modifier
.alpha(alpha)
.onFocusChanged { holdsFocus = it.hasFocus }
.let { base ->
if (visible) base else base.pointerInput(Unit) {
awaitPointerEventScope {
@@ -468,7 +474,10 @@ internal fun CarGlyph(icon: CarIcon?, modifier: Modifier = Modifier, standard: I
val white = remember(bitmap) { bitmap.isPlainWhite() }
// Untinted white artwork is a template-app habit from dark surfaces; it is coloured like
// text so it stays readable on a light one.
val tint = icon?.tint?.color(dark, content) ?: content.takeIf { white }
// An app's own tint is kept only if it can be seen: MAPS.ME asks for dark grey on dark
// buttons. Android Auto drops colours that fail contrast the same way.
val tint = icon?.tint?.color(dark, content)?.takeIf { it.readableOn(content) }
?: content.takeIf { white || icon?.tint != null }
Image(
bitmap, contentDescription = null, modifier = modifier,
colorFilter = tint?.let { ColorFilter.tint(it) },
@@ -587,16 +596,141 @@ internal fun PaneRows(pane: Pane?, modifier: Modifier = Modifier, autoStart: Boo
// Some apps pad a pane with a row of blank strings; drawn, it is just a gap.
val rows = pane?.rows.orEmpty().filterNot { it.isBlank() }
val actions = pane?.actions.orEmpty().filter { it.isDrawable() }
val card = PlaceCard.of(rows)
Column(modifier) {
rows.forEach { RowItem(it, large = true) }
card.heading?.let { PlaceHeading(it) }
if (card.buttons.isNotEmpty()) RowButtons(card.buttons, Modifier.padding(horizontal = 14.dp, vertical = 6.dp))
if (autoStart && actions.size == 1) {
AutoStartButton(actions.single(), Modifier.padding(horizontal = 14.dp, vertical = 10.dp))
} else {
ActionRow(actions, Modifier.padding(horizontal = 14.dp, vertical = 10.dp), large = true)
}
if (card.heading == null && card.buttons.isEmpty()) {
card.details.forEach { RowItem(it, large = true) }
} else {
card.details.forEach { DetailRow(it, card.heading?.title?.text().orEmpty()) }
}
}
}
/**
* A pane split the way a place page reads: a name on top, the things to do with it as buttons,
* then the facts. Apps send all three as plain rows, so the split is worked out from the rows'
* shape alone. A pane that doesn't fit the shape - all of it tappable, like a menu - gets no
* heading and no buttons, and draws exactly as a plain list of rows.
*/
private class PlaceCard(val heading: CarRow?, val buttons: List<CarRow>, val details: List<CarRow>) {
companion object {
fun of(rows: List<CarRow>): PlaceCard {
// A short tappable row with an icon and nothing under it is a button in all but name:
// MAPS.ME's "Let's go" and "Save".
val buttons = rows.filter {
it.onClickDelegate != null && it.toggle == null && it.image != null &&
it.texts.orEmpty().isEmpty() && it.title.text().length <= 24
}.takeIf { it.size in 1..4 && it.size < rows.size }.orEmpty()
// A first row that can't be tapped and has no icon is the place's name and kind.
val heading = rows.firstOrNull()?.takeIf {
buttons.isNotEmpty() && it.onClickDelegate == null && it.image == null && it.toggle == null
}
return PlaceCard(heading, buttons, rows - buttons.toSet() - listOfNotNull(heading).toSet())
}
}
}
@Composable
private fun PlaceHeading(row: CarRow) {
Column(Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 6.dp)) {
Text(row.title.text(), style = ChromeType.title, maxLines = 2, overflow = TextOverflow.Ellipsis)
row.texts.orEmpty().take(2).forEach {
Text(it.text(), style = ChromeType.body, color = MaterialTheme.colorScheme.onSurfaceVariant, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
}
/** Button-shaped rows side by side, the first one filled as the thing most drivers came to do. */
@Composable
private fun RowButtons(rows: List<CarRow>, modifier: Modifier) {
val scheme = MaterialTheme.colorScheme
Row(modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
rows.forEachIndexed { i, row ->
GsIconBox(
onClick = { row.onClickDelegate.click("row") },
modifier = Modifier.weight(1f),
colors = if (i == 0) GsColors(scheme.primary, scheme.onPrimary) else GsColors(scheme.surfaceContainerHighest, scheme.onSurface),
enabled = row.isEnabled,
shape = RoundedCornerShape(24.dp),
) {
Row(
Modifier.fillMaxWidth().padding(horizontal = 16.dp, vertical = 14.dp),
horizontalArrangement = Arrangement.Center,
verticalAlignment = Alignment.CenterVertically,
) {
CarGlyph(row.image, Modifier.size(24.dp))
Spacer(Modifier.width(8.dp))
Text(row.title.text(), style = ChromeType.body, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
}
}
}
/**
* One fact about a place, tighter than a list row: a small label over its value, or just the
* value when the row has no label. A phone number gets a Call button that asks first.
*/
@Composable
private fun DetailRow(row: CarRow, placeName: String) {
val context = LocalContext.current
val texts = row.texts.orEmpty().map { it.text() }.filter { it.isNotBlank() }
val label = row.title.text().takeIf { texts.isNotEmpty() }
val values = texts.ifEmpty { listOf(row.title.text()) }
val number = values.firstOrNull { it.isPhoneNumber() }
val canCall = number != null &&
context.checkSelfPermission(android.Manifest.permission.CALL_PHONE) == android.content.pm.PackageManager.PERMISSION_GRANTED
Row(
Modifier
.fillMaxWidth()
.let { if (row.onClickDelegate != null && row.isEnabled) it.clickable { row.onClickDelegate.click("row") } else it }
.padding(horizontal = 14.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
) {
row.image?.let {
CarGlyph(it, Modifier.size(24.dp))
Spacer(Modifier.width(12.dp))
}
Column(Modifier.weight(1f)) {
label?.let {
Text(it, style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurfaceVariant, maxLines = 1)
}
values.take(3).forEach {
Text(it, style = ChromeType.body, maxLines = 2, overflow = TextOverflow.Ellipsis)
}
}
if (canCall) {
Spacer(Modifier.width(8.dp))
GsIconBox(
onClick = { CallPrompt.ask(placeName.ifBlank { number!! }, number!!) },
colors = GsColors(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.onPrimary),
shape = RoundedCornerShape(24.dp),
) {
Row(Modifier.padding(horizontal = 16.dp, vertical = 10.dp), verticalAlignment = Alignment.CenterVertically) {
androidx.compose.material3.Icon(Icons.Filled.Call, null, Modifier.size(20.dp))
Spacer(Modifier.width(6.dp))
Text("Call", style = ChromeType.body)
}
}
}
ActionRow(row.actions.orEmpty())
}
}
/** A line that is only a phone number: digits with the usual separators, 7 to 15 digits long. */
private fun String.isPhoneNumber(): Boolean {
val t = trim()
if (!Regex("""^\+?[0-9 ().\-]+$""").matches(t)) return false
return t.count { it.isDigit() } in 7..15
}
@Composable
internal fun RowItem(row: CarRow, selection: RowSelection? = null, large: Boolean = false) {
val toggle = row.toggle
@@ -661,6 +795,17 @@ internal fun RowItem(row: CarRow, selection: RowSelection? = null, large: Boolea
/** A row's place in a single-choice list: whether it is the chosen one, and what picking it does. */
internal class RowSelection(val selected: Boolean, val onPick: () -> Unit)
/**
* Whether this colour stands out against the background that [content] was chosen for: at least
* 3:1, the WCAG minimum for icons. The background is taken as the opposite of the content colour.
*/
private fun Color.readableOn(content: Color): Boolean {
val background = if (content.luminanceIsDark()) Color.White else Color(0xFF121212)
val a = luminance() + 0.05f
val b = background.luminance() + 0.05f
return maxOf(a, b) / minOf(a, b) >= 3f
}
internal fun Color.luminanceIsDark(): Boolean =
(red * 0.299f + green * 0.587f + blue * 0.114f) < 0.5f
@@ -0,0 +1,94 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Close
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import app.seb3thehacker.gearslip.car.theme.GsIconButton
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/** What the assistant understands, shown on the car screen when the driver says "help". */
object VoiceHelp {
private val _shown = MutableStateFlow(false)
val shown: StateFlow<Boolean> = _shown.asStateFlow()
fun show() { _shown.value = true }
fun hide() { _shown.value = false }
/** Say this, and this happens. Kept in step with [AssistantCommands]. */
val COMMANDS = listOf(
"Navigate to…" to "Directions there. Add \"with Organic Maps\" to pick the app",
"Play…" to "A song, artist, or album in the open music app",
"Pause · Resume · Next · Previous" to "Control the music",
"What's playing" to "The song and artist",
"Call…" to "A contact by name",
"Open…" to "Any app, like Browser, Phone, or Screen sharing",
"Exit" to "Back to the car's own screen",
"Go home · Open apps · Settings" to "Move around Gearslip",
"What time is it · Weather" to "The time, or the forecast",
"Help" to "This list",
)
}
/** The command list over the car screen: read it at a glance, tap the cross or wait to dismiss. */
@Composable
fun VoiceHelpOverlay(modifier: Modifier = Modifier) {
val shown by VoiceHelp.shown.collectAsState()
if (!shown) return
// A list left up would cover the map for the rest of the drive.
LaunchedEffect(Unit) {
delay(30_000)
VoiceHelp.hide()
}
Surface(
color = MaterialTheme.colorScheme.surfaceContainerHigh,
contentColor = MaterialTheme.colorScheme.onSurface,
shape = MaterialTheme.shapes.extraLarge,
tonalElevation = 6.dp,
shadowElevation = 8.dp,
modifier = modifier.padding(16.dp).widthIn(max = 640.dp),
) {
Column(Modifier.padding(20.dp)) {
Row(Modifier.fillMaxWidth()) {
Text(
"You can say",
style = MaterialTheme.typography.titleLarge,
fontWeight = FontWeight.Bold,
modifier = Modifier.weight(1f).padding(top = 6.dp),
)
GsIconButton(Icons.Filled.Close, "Close", VoiceHelp::hide)
}
// Full width, so a drag anywhere on the card scrolls, not just one on the words.
Column(
Modifier.fillMaxWidth().verticalScroll(rememberScrollState()).padding(top = 8.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
VoiceHelp.COMMANDS.forEach { (say, does) ->
Column(Modifier.fillMaxWidth()) {
Text(say, style = MaterialTheme.typography.titleMedium, fontWeight = FontWeight.SemiBold)
Text(does, style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
}
}
}
}
@@ -12,6 +12,7 @@ import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.ui.unit.dp
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.getValue
@@ -19,6 +20,7 @@ import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.toArgb
import androidx.compose.ui.viewinterop.AndroidView
import app.seb3thehacker.gearslip.GearslipLog
import org.json.JSONObject
@@ -38,6 +40,7 @@ import org.json.JSONObject
fun WebApp(content: String) {
val navigator = LocalCarNavigator.current
val darkOutside = LocalDarkOutside.current
val ring = String.format("#%06X", MaterialTheme.colorScheme.primary.toArgb() and 0xFFFFFF)
var web by remember { mutableStateOf<WebView?>(null) }
var fieldText by remember { mutableStateOf<String?>(null) }
@@ -56,7 +59,7 @@ fun WebApp(content: String) {
Column(Modifier.fillMaxSize()) {
// Back steps through the page's history first (the interceptor above), then leaves.
ScreenHeader("Web", Modifier.padding(horizontal = 16.dp, vertical = 8.dp))
ScreenHeader("Browser", Modifier.padding(horizontal = 16.dp, vertical = 8.dp))
AndroidView(
modifier = Modifier.weight(1f).fillMaxSize(),
factory = { context ->
@@ -65,7 +68,7 @@ fun WebApp(content: String) {
settings.domStorageEnabled = true
settings.loadWithOverviewMode = true
settings.useWideViewPort = true
webViewClient = LoggingClient()
webViewClient = LoggingClient(ring)
addJavascriptInterface(
FieldFocusBridge(onFocus = { fieldText = it }, onBlur = { fieldText = null }),
"GearslipField",
@@ -99,10 +102,11 @@ fun WebApp(content: String) {
}
}
private class LoggingClient : WebViewClient() {
private class LoggingClient(private val ring: String) : WebViewClient() {
override fun onPageFinished(view: WebView?, url: String?) {
GearslipLog.i("car web page loaded: $url")
view?.evaluateJavascript(FOCUS_TRACKING_JS, null)
view?.evaluateJavascript(focusRingJs(ring), null)
}
// onPageFinished fires for error pages too, so without this a blocked load (missing
@@ -170,6 +174,20 @@ private const val SUBMIT_FIELD_JS = """
})();
"""
/**
* The same ring the rest of the car UI shows, on whatever link or field the knob is on. Many
* sites hide the browser's own focus outline, which leaves a knob driver with no idea where they are.
*/
private fun focusRingJs(color: String) = """
(function() {
if (document.getElementById('gearslip-focus-ring')) return;
var style = document.createElement('style');
style.id = 'gearslip-focus-ring';
style.textContent = ':focus-visible { outline: 3px solid $color !important; outline-offset: 2px !important; }';
(document.head || document.documentElement).appendChild(style);
})();
"""
/** Reports focus/blur on text inputs, textareas and contenteditable elements to [FieldFocusBridge]. */
private const val FOCUS_TRACKING_JS = """
(function() {
@@ -50,6 +50,8 @@ enum class HoldMarkerStyle {
* colour, so the peel's flap matches whatever it sits on. Round and pill-shaped controls get the
* dot whatever the theme says: a circle has no corner to peel.
*/
private const val SHOW_HOLD_DOT = false
@Composable
internal fun BoxScope.GsHoldMarker(style: HoldMarkerStyle, container: Color, shape: Shape, press: InteractionSource) {
if (style == HoldMarkerStyle.NONE) return
@@ -68,8 +70,9 @@ internal fun BoxScope.GsHoldMarker(style: HoldMarkerStyle, container: Color, sha
val side = min(size.width, size.height)
val r = (shape as? CornerBasedShape)?.topEnd?.toPx(size, this) ?: 0f
val round = r >= side * 0.4f
// The dot is off until it's redesigned: flat themes and round controls show nothing.
if (style == HoldMarkerStyle.DOT || round) {
drawHoldDot(r, side, ink)
if (SHOW_HOLD_DOT) drawHoldDot(r, side, ink)
} else {
// Drawn for the bottom-right corner, then flipped up to the top-right.
scale(1f, -1f, pivot = center) {
@@ -112,7 +112,7 @@ object GsThemes {
control = ControlLook(holdMarker = HoldMarkerStyle.DOT),
)
val default = Tactile
val default = Flat
val builtIn = listOf(Tactile, Keycap, Flat)
fun byId(id: String?): GsTheme = builtIn.firstOrNull { it.id == id } ?: default
@@ -0,0 +1,98 @@
package app.seb3thehacker.gearslip.host
import android.os.SystemClock
import android.text.Spanned
import androidx.car.app.model.CarText
import androidx.car.app.model.DistanceSpan
import androidx.car.app.model.ItemList
import androidx.car.app.model.ListTemplate
import androidx.car.app.model.PaneTemplate
import androidx.car.app.model.PlaceListMapTemplate
import androidx.car.app.model.Row
import androidx.car.app.model.SearchTemplate
import androidx.car.app.model.Template
import androidx.car.app.navigation.model.MapTemplate
import androidx.car.app.navigation.model.MapWithContentTemplate
import androidx.car.app.navigation.model.NavigationTemplate
import androidx.car.app.navigation.model.PlaceListNavigationTemplate
import androidx.car.app.navigation.model.RoutePreviewNavigationTemplate
import app.seb3thehacker.gearslip.GearslipLog
/**
* "Navigate to X" the way Android Auto does it: straight onto a route, no list to pick from.
*
* Android Auto looks the place up on Google's servers first. Gearslip stays on the phone, so it
* asks the map app to search, then taps the nearest result in the app's list for the driver.
* It stops at the app's start screen, so the driver sees where it's going and taps Start. It
* gives up after [WINDOW_MS], or as soon as the driver taps anything themselves.
*/
internal class AutoRoute {
private val log = GearslipLog.tagged("route")
private var until = 0L
private var pickedRow = false
/** A destination was just sent to the map app. */
fun arm() {
armedAt = SystemClock.elapsedRealtime()
until = armedAt + WINDOW_MS
pickedRow = false
}
fun disarm() {
until = 0L
}
private var armedAt = 0L
private val armed get() = SystemClock.elapsedRealtime() < until && driverTappedAt < armedAt
/** Each screen the map app sends while armed: tap the next step if it's there. */
@Suppress("DEPRECATION") // Apps still send the older navigation templates.
fun onTemplate(template: Template) {
if (!armed) return
when (template) {
is NavigationTemplate -> disarm()
// The start screen: the driver takes it from here.
is RoutePreviewNavigationTemplate -> arrived()
is MapWithContentTemplate -> onTemplate(template.contentTemplate)
is PaneTemplate -> arrived()
is MapTemplate -> if (template.pane != null) arrived() else template.itemList?.let(::onList)
is SearchTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
is ListTemplate -> if (!template.isLoading) template.singleList?.let(::onList)
is PlaceListMapTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
is PlaceListNavigationTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
}
}
/** Search results: the nearest place, or the first when the app gives no distances. */
private fun onList(list: ItemList) {
if (pickedRow) return
val places = list.items.filterIsInstance<Row>()
.filter { it.isEnabled && it.onClickDelegate != null && !it.isBrowsable }
if (places.isEmpty()) return
val nearest = places.minByOrNull { it.meters() ?: Double.MAX_VALUE } ?: return
pickedRow = true
log.i("picking the nearest of ${places.size} results")
nearest.onClickDelegate.click(AUTO_LABEL)
}
private fun arrived() {
if (pickedRow) log.i("at the start screen")
disarm()
}
/** The first distance the app attached to this row's title or text lines. */
private fun Row.meters(): Double? = (listOfNotNull(title) + texts).firstNotNullOfOrNull { it.distanceMeters() }
private fun CarText.distanceMeters(): Double? {
val chars = toCharSequence() as? Spanned ?: return null
return chars.getSpans(0, chars.length, DistanceSpan::class.java).firstOrNull()?.distance?.meters()
}
companion object {
/** Taps sent by this class carry this label, so they don't count as the driver's. */
const val AUTO_LABEL = "auto-route"
/** When the driver last tapped one of the app's buttons; that hands control back. */
@Volatile var driverTappedAt = 0L
private const val WINDOW_MS = 30_000L
}
}
@@ -78,5 +78,7 @@ object CarAppCatalog {
CarAppService.CATEGORY_SETTINGS_APP,
CarAppService.CATEGORY_IOT_APP,
CarAppService.CATEGORY_WEATHER_APP,
// Newer apps declare MEDIA, but our AndroidX 1.7 dependency has no constant for it.
"androidx.car.app.category.MEDIA",
)
}
@@ -1,6 +1,7 @@
package app.seb3thehacker.gearslip.host
import app.seb3thehacker.gearslip.car.CarEnvironment
import app.seb3thehacker.gearslip.car.CarToasts
import android.content.ComponentName
import android.content.Context
import android.content.Intent
@@ -9,6 +10,7 @@ import android.content.res.Configuration
import android.graphics.Rect
import android.location.Location
import android.net.Uri
import android.os.Build
import android.os.Handler
import android.os.IBinder
import android.os.Looper
@@ -80,6 +82,8 @@ class CarAppConnection(private val context: Context) {
val template: StateFlow<Template?> = _template.asStateFlow()
private val main = Handler(Looper.getMainLooper())
/** Captured by each host Binder so callbacks queued by a replaced app cannot show a toast. */
private var toastOwner = Any()
private var carApp: ICarApp? = null
private var appManager: IAppManager? = null
@@ -139,7 +143,11 @@ class CarAppConnection(private val context: Context) {
// Replaced by another connect (or a disconnect) while waiting on the location service.
if (binding !== connection) return@bind
var bindError: Throwable? = null
val flags = Context.BIND_AUTO_CREATE or LocationKeepAlive.BIND_INCLUDE_CAPABILITIES
// BIND_ALLOW_ACTIVITY_STARTS lets the app start its own location service while
// Gearslip's car window is showing. Since Android 14 a bound app only inherits that
// from a visible host that opts in, and MapQuest crashes as navigation starts without it.
val flags = Context.BIND_AUTO_CREATE or LocationKeepAlive.BIND_INCLUDE_CAPABILITIES or
(if (Build.VERSION.SDK_INT >= 34) Context.BIND_ALLOW_ACTIVITY_STARTS else 0)
val bound = runCatching {
// Without BIND_FOREGROUND_SERVICE, an app bound by a foreground service (which is
// all Gearslip is once the phone screen leaves it) only reaches "important
@@ -215,6 +223,7 @@ class CarAppConnection(private val context: Context) {
fun disconnect() = disconnect(keepLocation = false)
private fun disconnect(keepLocation: Boolean) {
clearToasts()
lent?.let { sendSurfaceDestroyed(it) }
lent = null
val carApp = this.carApp
@@ -231,6 +240,10 @@ class CarAppConnection(private val context: Context) {
this.carApp = null
appManager = null
navigationManager = null
navigating = false
main.removeCallbacks(sendReroute)
reroute = null
autoRoute.disarm()
_panMode.value = false
surfaceCallback = null
component = null
@@ -281,34 +294,66 @@ class CarAppConnection(private val context: Context) {
private fun onHandshakeAccepted() {
val app = carApp ?: return
GearslipLog.i("host: handshake accepted; creating the app")
val intent = pendingDestination?.let { navigateIntent(it) }
val intent = pendingDestination?.let { autoRoute.arm(); navigateIntent(it) }
?: Intent(Intent.ACTION_MAIN).setComponent(component)
pendingDestination = null
call("onAppCreate", onValue = { onCreated() }) {
app.onAppCreate(carHost, intent, carConfiguration(), it)
app.onAppCreate(createCarHost(toastOwner), intent, carConfiguration(), it)
}
}
/**
* "navigate to X" deep link, the same one Android Auto's own Assistant integration sends -
* [ACTION_NAVIGATE] with the destination as a `geo:` search query, rather than anything
* app-specific.
* [ACTION_NAVIGATE] with the destination as a `geo:0,0` search query, rather than anything
* app-specific. The app searches near its own location; [AutoRoute] then picks the nearest
* result. Coordinates in the link would mean "go exactly here": Organic Maps routed to the
* phone's own position when it was given them.
*/
private fun navigateIntent(destination: String) =
private fun navigateIntent(destination: String): Intent =
Intent(CarContext.ACTION_NAVIGATE, Uri.parse("geo:0,0?q=" + Uri.encode(destination)))
.setComponent(component)
/**
* Sends a fresh destination to an app that's already running, instead of tearing it down
* and rebuilding the whole session - see [app.seb3thehacker.gearslip.car.CarServices.connectNav].
*
* An app already on a route ends it first: Organic Maps ignored a new destination mid-route
* and kept the old directions. The new one goes once the app says the route ended, or after
* [REROUTE_WAIT_MS] if it never does.
*/
fun navigateTo(destination: String) {
if (carApp == null) return
main.removeCallbacks(sendReroute)
if (!navigating) {
reroute = null
sendDestination(destination)
return
}
reroute = destination
stopNavigating()
main.postDelayed(sendReroute, REROUTE_WAIT_MS)
}
private fun sendDestination(destination: String) {
val app = carApp ?: return
autoRoute.arm()
call("onNewIntent", onValue = {}) {
app.onNewIntent(navigateIntent(destination), it)
}
}
private val autoRoute = AutoRoute()
/** A destination waiting for the current route to end; see [navigateTo]. */
private var reroute: String? = null
private val sendReroute = Runnable {
reroute?.let { reroute = null; sendDestination(it) }
}
/** Whether the app has said it's on a route, between navigationStarted and navigationEnded. */
@Volatile var navigating = false
private set
private fun onCreated() {
val app = carApp ?: return
GearslipLog.i("host: created; starting")
@@ -380,6 +425,7 @@ class CarAppConnection(private val context: Context) {
GearslipLog.i("host: template = ${template.javaClass.simpleName}")
_template.value = template
update { it.copy(template = template.javaClass.simpleName) }
autoRoute.onTemplate(template)
}) { manager.getTemplate(it) }
}
@@ -558,14 +604,19 @@ class CarAppConnection(private val context: Context) {
// --- host interfaces the app calls back into -----------------------------------------
private val appHost = object : IAppHost.Stub() {
private fun createAppHost(owner: Any) = object : IAppHost.Stub() {
override fun invalidate() {
GearslipLog.i("host: app invalidated its template")
main.post { requestTemplate() }
}
override fun showToast(text: CharSequence?, duration: Int) {
GearslipLog.i("host: toast \"$text\"")
val message = text?.toString() ?: return
main.post {
if (toastOwner !== owner) return@post
GearslipLog.i("host: toast \"$message\"")
CarToasts.show(owner, message, duration)
}
}
/**
@@ -587,10 +638,24 @@ class CarAppConnection(private val context: Context) {
override fun openMicrophone(request: Bundleable?): Bundleable? = null
}
private val _trip = MutableStateFlow<androidx.car.app.navigation.model.Trip?>(null)
/** The route as the app tells clusters about it: steps and how far to each. Null when not navigating. */
val trip: StateFlow<androidx.car.app.navigation.model.Trip?> = _trip.asStateFlow()
private val navigationHost = object : INavigationHost.Stub() {
override fun navigationStarted() = GearslipLog.i("host: navigation started")
override fun navigationEnded() = GearslipLog.i("host: navigation ended")
override fun updateTrip(trip: Bundleable?) = Unit
override fun navigationStarted() {
GearslipLog.i("host: navigation started")
navigating = true
}
override fun navigationEnded() {
GearslipLog.i("host: navigation ended")
navigating = false
_trip.value = null
main.post { if (reroute != null) { main.removeCallbacks(sendReroute); sendReroute.run() } }
}
override fun updateTrip(trip: Bundleable?) {
_trip.value = runCatching { trip?.get() as? androidx.car.app.navigation.model.Trip }.getOrNull()
}
}
/**
@@ -656,7 +721,9 @@ class CarAppConnection(private val context: Context) {
override fun updateSuggestions(suggestions: Bundleable?) = Unit
}
private val carHost = object : ICarHost.Stub() {
private fun createCarHost(owner: Any) = object : ICarHost.Stub() {
private val appHost = createAppHost(owner)
override fun getHost(type: String?): IBinder? = when (type) {
CarContext.APP_SERVICE -> appHost.asBinder()
CarContext.NAVIGATION_SERVICE -> navigationHost.asBinder()
@@ -745,10 +812,16 @@ class CarAppConnection(private val context: Context) {
}
private fun fail(phase: Phase, detail: String) {
clearToasts()
GearslipLog.e("host: $detail")
update { it.copy(phase = phase, detail = detail) }
}
private fun clearToasts() {
CarToasts.clear(toastOwner)
toastOwner = Any()
}
private fun update(block: (Status) -> Status) {
_status.value = block(_status.value)
}
@@ -757,6 +830,9 @@ class CarAppConnection(private val context: Context) {
/** The newest template API this host claims to understand. */
const val HOST_API_LEVEL = CarAppApiLevels.LEVEL_8
/** How long a new destination waits for the old route to end; see [navigateTo]. */
const val REROUTE_WAIT_MS = 1_500L
/** Context.BIND_FOREGROUND_SERVICE; the constant itself is hidden from the SDK. */
const val BIND_FOREGROUND_SERVICE = 0x04000000
}
@@ -141,7 +141,7 @@ fun Distance?.display(): String {
return "%.${decimals}f $unit".format(distance.displayDistance)
}
private fun Distance.meters(): Double = displayDistance * when (displayUnit) {
internal fun Distance.meters(): Double = displayDistance * when (displayUnit) {
Distance.UNIT_KILOMETERS, Distance.UNIT_KILOMETERS_P1 -> 1000.0
Distance.UNIT_MILES, Distance.UNIT_MILES_P1 -> 1609.344
Distance.UNIT_FEET -> 0.3048
@@ -195,6 +195,7 @@ fun androidx.car.app.model.DateTimeWithZone?.clockTime(): String {
*/
fun OnClickDelegate?.click(label: String = "action") {
val delegate = this ?: return
if (label != AutoRoute.AUTO_LABEL) AutoRoute.driverTappedAt = android.os.SystemClock.elapsedRealtime()
runCatching { delegate.sendClick(doneCallback("a $label tap")) }
.onFailure { GearslipLog.w("host: could not send a $label tap: ${it.message}") }
}
@@ -1,5 +1,9 @@
package app.seb3thehacker.gearslip.host
import android.content.Context
import android.content.pm.PackageManager
import java.security.MessageDigest
/**
* Apps that have been run through Gearslip from start to finish and behave. The launcher marks
* them so a driver can pick something that works without finding out at the wheel.
@@ -18,6 +22,8 @@ object KnownApps {
"com.vivi.vivimusic", // Vivi Music: browse, play, seek, works well
"com.metrolist.music", // Metrolist: browse, play, works
"nl.flitsmeister", // Flitsmeister: map, routing, alerts - works well
"com.mapquest.android.ace", // MapQuest: map, routing, navigation
"com.mapswithme.maps.pro", // MAPS.ME: map, search, place pages, navigation
)
fun works(packageName: String): Boolean = packageName in working
@@ -63,7 +69,30 @@ object KnownApps {
"com.generalmagic.magicearth", // Magic Earth: never gets past connecting
)
fun isBroken(packageName: String): Boolean = packageName in broken
fun isBroken(context: Context, packageName: String): Boolean =
packageName in broken && !patched(context, packageName)
/**
* The certificate each broken app's own publisher signs it with. A copy signed with another
* key has been patched to accept Gearslip, like Waze with the waze-gearslip Morphe patch, and
* works.
*/
private val officialSigners = mapOf(
"com.waze" to "03637f6c5d8f604e6fdb79a6ffbfa578de4e318f8da22fc6106665247f8807d7",
)
/** Whether the installed copy is signed by someone other than its publisher. */
private fun patched(context: Context, packageName: String): Boolean {
val official = officialSigners[packageName] ?: return false
val signers = runCatching {
context.packageManager.getPackageInfo(packageName, PackageManager.GET_SIGNING_CERTIFICATES)
.signingInfo?.apkContentsSigners
}.getOrNull() ?: return false
val sha256 = MessageDigest.getInstance("SHA-256")
return signers.isNotEmpty() && signers.none { cert ->
sha256.digest(cert.toByteArray()).joinToString("") { "%02x".format(it) } == official
}
}
/**
* Shown on the launcher with a yellow mark: starts and can be used, but not cleanly enough to
@@ -71,7 +100,6 @@ object KnownApps {
*/
private val partial = setOf(
"app.vela", // Vela Maps: connects and draws, but rough enough not to call it working yet
"com.mapquest.android.ace", // MapQuest: map, search and routing run, but not cleanly
"com.spotify.music", // Spotify: plays through Gearslip's player, but the app has to be opened on the phone first
"deezer.android.app", // Deezer: the same - it has to be opened on the phone first
)
@@ -40,7 +40,19 @@ class MediaEntry(
)
/** [icon] is a drawable id inside the media app's own package, not Gearslip's; 0 for none. */
class CustomAction(val id: String, val name: String, val extras: Bundle?, val icon: Int = 0)
class CustomAction(val id: String, val name: String, val extras: Bundle?, val icon: Int = 0, val iconName: String = "") {
/**
* Whether a toggle like heart or shuffle is on, read from the icon's name - Tidal swaps
* ic_favorite for ic_favorite_filled and ic_shuffle for ic_shuffle_active, and keeps the
* button's name the same. Whole words only, so "inactive" doesn't count as "active".
*/
val on: Boolean
get() = iconName.lowercase().split('_', '.', '-').any { it in ON_WORDS }
private companion object {
val ON_WORDS = setOf("filled", "fill", "active", "on", "selected", "checked", "enabled", "solid", "one", "all")
}
}
/** One entry in the app's own queue - not its library, whatever it has lined up to play next. */
class QueueTrack(
@@ -156,6 +168,9 @@ class CarMedia(private val context: Context) {
private val main = Handler(Looper.getMainLooper())
private var connecting: MediaApp? = null
/** The music app Gearslip is attached to, for the assistant to name; null when there's none. */
val appLabel: String? get() = connecting?.label?.takeIf { controller != null }
private val _phase = MutableStateFlow(Phase.IDLE)
val phase: StateFlow<Phase> = _phase.asStateFlow()
@@ -225,7 +240,11 @@ class CarMedia(private val context: Context) {
private fun refreshModes() {
val c = compat
_modes.value = if (c == null) Modes() else Modes(shuffle = c.shuffleMode, repeat = c.repeatMode)
val modes = if (c == null) Modes() else Modes(shuffle = c.shuffleMode, repeat = c.repeatMode)
if (modes.shuffle != _modes.value.shuffle || modes.repeat != _modes.value.repeat) {
GearslipLog.i("media: shuffle mode ${modes.shuffle}, repeat mode ${modes.repeat}, rating type ${controller?.ratingType}")
}
_modes.value = modes
}
private var subscribed: String? = null
private var rootId: String? = null
@@ -236,6 +255,12 @@ class CarMedia(private val context: Context) {
connecting = app
_phase.value = Phase.CONNECTING
_browse.value = BrowseState()
// Known to refuse its library (Spotify): asking only costs the seconds it takes to say no,
// so go straight to its playback session.
if (KnownApps.playerWorks(app.component.packageName)) {
attachToSession(app)
return
}
val mb = MediaBrowser(context, app.component, object : MediaBrowser.ConnectionCallback() {
override fun onConnected() = this@CarMedia.onConnected(app)
override fun onConnectionFailed() {
@@ -248,10 +273,20 @@ class CarMedia(private val context: Context) {
}
}, null)
browser = mb
GearslipLog.i("media: connecting to ${app.label}'s library")
runCatching { mb.connect() }.onFailure {
GearslipLog.e("media: connect threw", it)
_phase.value = Phase.REJECTED
}
// An app that never answers would leave the player spinning, and every later tap on it
// would only reopen that spinner. Give up on the library and drive its playback instead.
main.postDelayed({
if (browser !== mb || _phase.value != Phase.CONNECTING) return@postDelayed
GearslipLog.w("media: ${app.label} didn't answer in ${CONNECT_WAIT_MS / 1000} s, using its playback session")
runCatching { mb.disconnect() }
browser = null
attachToSession(app)
}, CONNECT_WAIT_MS)
}
/**
@@ -273,7 +308,12 @@ class CarMedia(private val context: Context) {
return
}
if (session == null) {
val limit = if (launched) SESSION_RETRIES_LAUNCHED else SESSION_RETRIES
// A known session-only app that isn't running won't start by waiting: say so at once.
val limit = when {
launched -> SESSION_RETRIES_LAUNCHED
KnownApps.playerWorks(app.component.packageName) -> 0
else -> SESSION_RETRIES
}
// The app only publishes its session once its service has started; give it a moment.
if (attempt < limit) {
main.postDelayed({ attachToSession(app, attempt + 1, launched) }, SESSION_RETRY_MS)
@@ -342,16 +382,39 @@ class CarMedia(private val context: Context) {
speed = state?.playbackSpeed ?: 1f,
updatedAt = state?.lastPositionUpdateTime ?: 0,
actions = state?.actions ?: 0,
custom = state?.customActions.orEmpty().map { CustomAction(it.action, it.name.toString(), it.extras, it.icon) },
custom = state?.customActions.orEmpty().map { CustomAction(it.action, it.name.toString(), it.extras, it.icon, iconName(it.icon)) },
error = state?.errorMessage?.toString(),
activeQueueItemId = state?.activeQueueItemId ?: MediaSession.QueueItem.UNKNOWN_ID.toLong(),
heart = if (controller?.ratingType == Rating.RATING_HEART) {
metadata?.getRating(MediaMetadata.METADATA_KEY_USER_RATING)?.takeIf { it.isRated }?.hasHeart() ?: false
} else null,
)
logButtons()
checkSearch()
}
private var appResources: Pair<String, android.content.res.Resources?>? = null
/** The icon's resource name in the app's package, or "" when it can't be read. */
private fun iconName(icon: Int): String {
val pkg = controller?.packageName ?: return ""
if (appResources?.first != pkg) {
appResources = pkg to runCatching { context.packageManager.getResourcesForApplication(pkg) }.getOrNull()
}
val res = appResources?.second ?: return ""
return runCatching { res.getResourceEntryName(icon) }.getOrDefault("")
}
private var loggedButtons: String? = null
/** The app's own buttons, by id, name and icon, each time they change. */
private fun logButtons() {
val text = _now.value.custom.joinToString { a -> "${a.id} \"${a.name}\" icon=${a.iconName.ifEmpty { a.icon.toString() }}" }
if (text == loggedButtons) return
loggedButtons = text
GearslipLog.i("media: app buttons: ${text.ifEmpty { "none" }}")
}
// --- browsing ------------------------------------------------------------------------
private fun open(parentId: String) {
@@ -419,6 +482,12 @@ class CarMedia(private val context: Context) {
val controls = controller?.transportControls ?: return
if (_now.value.playing) controls.pause() else controls.play()
}
/**
* Pauses whatever the app thinks it's doing. Voice pauses the music to listen, so by the time
* "pause" is heard the app already looks stopped; a toggle did nothing and the song came back
* as soon as the microphone let go. A real pause tells the app to stay stopped.
*/
fun pause() = controller?.transportControls?.pause()
fun next() = controller?.transportControls?.skipToNext()
fun previous() = controller?.transportControls?.skipToPrevious()
fun seek(ms: Long) = controller?.transportControls?.seekTo(ms)
@@ -454,7 +523,7 @@ class CarMedia(private val context: Context) {
}
val component = searchComponent
if (component == null) {
_results.value = Results(q, emptyList(), supported = false)
noLibrarySearch(q)
return
}
resultsQuery = q
@@ -476,13 +545,13 @@ class CarMedia(private val context: Context) {
override fun onConnectionFailed() {
GearslipLog.w("media: no library connection for search")
searchBrowser = null
resultsQuery?.let { _results.value = Results(it, emptyList(), supported = false) }
resultsQuery?.let { noLibrarySearch(it) }
}
}, null)
searchBrowser = created
runCatching { created.connect() }.onFailure {
searchBrowser = null
_results.value = Results(q, emptyList(), supported = false)
noLibrarySearch(q)
}
}
@@ -492,18 +561,42 @@ class CarMedia(private val context: Context) {
override fun onSearchResult(query: String, extras: Bundle?, items: List<MediaBrowserCompat.MediaItem>) {
if (query != resultsQuery) return
GearslipLog.i("media: library search found ${items.size}")
_results.value = Results(query, items.map(::entryOfCompat))
val entries = items.map(::entryOfCompat)
if (query == playFirstFor) {
playFirstFor = null
val first = entries.firstOrNull { it.playable } ?: entries.firstOrNull()
if (first == null) {
clearResults()
search(query)
} else {
GearslipLog.i("media: playing the library's best match")
clearResults()
searchBefore = _now.value
_search.value = Search(query)
main.removeCallbacks(searchTimeout)
controller?.transportControls?.playFromMediaId(first.id, null)
main.postDelayed(searchTimeout, SEARCH_WAIT_MS)
}
return
}
_results.value = Results(query, entries)
}
override fun onError(query: String, extras: Bundle?) {
if (query != resultsQuery) return
if (query == playFirstFor) {
playFirstFor = null
clearResults()
search(query)
return
}
GearslipLog.i("media: the app has no library search")
_results.value = Results(query, emptyList(), supported = false)
}
})
}.onFailure {
GearslipLog.w("media: library search failed: ${it.message}")
_results.value = Results(q, emptyList(), supported = false)
noLibrarySearch(q)
}
}
@@ -520,6 +613,36 @@ class CarMedia(private val context: Context) {
)
}
/** The library can't search: a voice "play X" falls back to the app's own search; a typed one says so. */
private fun noLibrarySearch(q: String) {
if (q == playFirstFor) {
playFirstFor = null
clearResults()
search(q)
} else {
_results.value = Results(q, emptyList(), supported = false)
}
}
/** A voice "play X" waiting on the library search, to play its first match instead of listing them. */
private var playFirstFor: String? = null
/**
* Plays the best match for [query], for voice. Asks the app's library first and plays its top
* result, which apps like ViVi honour where they ignore [search]; without a library search,
* falls back to [search].
*/
fun playBestMatch(query: String) {
val q = query.trim()
if (q.isEmpty()) return
if (!canSearchLibrary) {
search(q)
return
}
playFirstFor = q
searchLibrary(q)
}
/** The connected app's service, for the search connection; null when nothing is connected. */
private var searchComponent: android.content.ComponentName? = null
private var searchBrowser: MediaBrowserCompat? = null
@@ -637,6 +760,7 @@ class CarMedia(private val context: Context) {
}
private companion object {
const val CONNECT_WAIT_MS = 8_000L
const val SESSION_RETRIES = 6
const val SESSION_RETRY_MS = 500L
/** After opening a cold app, how long to wait before the first look for its session. */
@@ -0,0 +1,119 @@
package app.seb3thehacker.gearslip.media
import android.graphics.Bitmap
import android.media.session.PlaybackState
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
import app.seb3thehacker.gearslip.car.CarServices
import java.io.ByteArrayOutputStream
/**
* Tells the car what's playing, so its own screens show it: the instrument cluster, the media
* source screen, the steering wheel display. Android Auto does the same over the car's
* MediaPlaybackStatusService (Service field 9):
*
* -> MediaPlaybackMetadata { song = 1; artist = 2; album = 3; album_art = 4; duration_seconds = 6 }
* -> MediaPlaybackStatus { state = 1 (1 stopped, 2 playing, 3 paused); media_source = 2;
* playback_seconds = 3; shuffle = 4; repeat = 5; repeat_one = 6 }
*
* Read once a second from the media app Gearslip is driving. The track goes only when it changes;
* the status goes every second while playing, so the car's position counts along.
*/
class CarMediaStatus(
val channelId: Int,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
) {
@Volatile private var running = false
private var sentTrack: String? = null
private var sentState = -1
private var inputsLogged = 0
fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
GearslipLog.i("media status: <- ChannelOpenResponse: status=$status")
if (status == 0) start()
}
// The car's own buttons for the media source (play, next...). Logged to learn what
// this car sends; the steering wheel keys already arrive on the input channel.
MSG_INPUT -> if (inputsLogged++ < 10) GearslipLog.i("media status: <- input from the car:\n" + Protobuf.describe(body))
else -> GearslipLog.i("media status: <- unhandled message id=$messageId (${body.size} bytes)")
}
}
private fun start() {
if (running) return
running = true
Thread({
while (running) {
runCatching { tick() }.onFailure { GearslipLog.w("media status: ${it.message}") }
Thread.sleep(TICK_MS)
}
}, "gearslip-media-status").start()
}
fun stop() {
running = false
}
private fun tick() {
val now = CarServices.mediaOrNull()?.now?.value
val active = now != null && now.isActive
val state = when {
!active -> STATE_STOPPED
now!!.state == PlaybackState.STATE_PAUSED -> STATE_PAUSED
else -> STATE_PLAYING
}
if (active) sendTrack(now!!)
// Nothing to say while stopped, beyond saying so once.
if (state == STATE_STOPPED && sentState == STATE_STOPPED) return
if (state == STATE_PAUSED && sentState == STATE_PAUSED) return
val source = CarServices.mediaApp.value?.label ?: "Gearslip"
val seconds = if (active) (now!!.currentPosition() / 1000).coerceAtLeast(0) else 0
sendOnChannel(
MSG_STATUS,
Protobuf.varintField(1, state.toLong()) +
Protobuf.stringField(2, source) +
Protobuf.varintField(3, seconds) +
Protobuf.varintField(4, 0L) +
Protobuf.varintField(5, 0L) +
Protobuf.varintField(6, 0L),
)
if (state != sentState) GearslipLog.i("media status: -> ${STATE_NAMES[state]} ($source)")
sentState = state
}
private fun sendTrack(now: NowPlaying) {
val key = "${now.title}\u0000${now.artist}\u0000${now.album}\u0000${now.durationMs}\u0000${now.art?.generationId}"
if (key == sentTrack) return
sentTrack = key
var body = Protobuf.stringField(1, now.title) +
Protobuf.stringField(2, now.artist) +
Protobuf.stringField(3, now.album)
now.art?.let { art -> artBytes(art)?.let { body += Protobuf.bytesField(4, it) } }
if (now.durationMs > 0) body += Protobuf.varintField(6, now.durationMs / 1000)
sendOnChannel(MSG_METADATA, body)
GearslipLog.i("media status: -> track \"${now.title}\" by ${now.artist.ifBlank { "?" }}")
}
/** Album art as a small PNG: car screens show it at thumbnail size, and the link is shared with video. */
private fun artBytes(art: Bitmap): ByteArray? = runCatching {
val scale = minOf(1f, ART_PX.toFloat() / maxOf(art.width, art.height))
val small = if (scale < 1f) Bitmap.createScaledBitmap(art, (art.width * scale).toInt(), (art.height * scale).toInt(), true) else art
ByteArrayOutputStream().also { small.compress(Bitmap.CompressFormat.PNG, 100, it) }.toByteArray()
}.getOrNull()
companion object {
private const val TICK_MS = 1_000L
private const val ART_PX = 256
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_STATUS = 32769
private const val MSG_INPUT = 32770
private const val MSG_METADATA = 32771
private const val STATE_STOPPED = 1
private const val STATE_PLAYING = 2
private const val STATE_PAUSED = 3
private val STATE_NAMES = mapOf(1 to "stopped", 2 to "playing", 3 to "paused")
}
}
@@ -30,7 +30,7 @@ class GearslipNotificationListener : NotificationListenerService() {
override fun onNotificationRemoved(sbn: StatusBarNotification) = CarNotifications.gone(sbn.key)
private fun ingest(sbn: StatusBarNotification, quiet: Boolean) {
// Gearslip's own notifications are skipped, except a dev build's test message.
// Gearslip's own notifications are skipped, except a debug build's test message.
if (sbn.packageName == packageName &&
!(app.seb3thehacker.gearslip.BuildConfig.DEBUG && sbn.notification.channelId == TestMessage.CHANNEL)
) return
@@ -13,7 +13,7 @@ import androidx.core.app.RemoteInput
import app.seb3thehacker.gearslip.GearslipLog
/**
* Dev builds only: posts a chat message from Gearslip to itself, shaped like Signal's or
* Debug builds only: posts a chat message from Gearslip to itself, shaped like Signal's or
* WhatsApp's (MessagingStyle, a Reply action, a Mark as read action), so popups and voice reply
* can be tried without asking someone to text you. The listener lets these through only in a
* debug build; everything else Gearslip posts is still ignored.
@@ -4,7 +4,6 @@ import android.Manifest
import android.annotation.SuppressLint
import android.content.Context
import android.content.pm.PackageManager
import android.content.res.AssetManager
import android.media.AudioFormat
import android.media.AudioManager
import android.media.AudioRecord
@@ -15,38 +14,50 @@ import android.os.Looper
import android.os.SystemClock
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.audio.CarMic
import java.util.Locale
import com.k2fsa.sherpa.onnx.EndpointConfig
import com.k2fsa.sherpa.onnx.EndpointRule
import com.k2fsa.sherpa.onnx.FeatureConfig
import com.k2fsa.sherpa.onnx.OfflineCanaryModelConfig
import com.k2fsa.sherpa.onnx.OfflineModelConfig
import com.k2fsa.sherpa.onnx.OfflineRecognizer
import com.k2fsa.sherpa.onnx.OfflineRecognizerConfig
import com.k2fsa.sherpa.onnx.OfflineTransducerModelConfig
import com.k2fsa.sherpa.onnx.OnlineModelConfig
import com.k2fsa.sherpa.onnx.OnlineRecognizer
import com.k2fsa.sherpa.onnx.OnlineRecognizerConfig
import com.k2fsa.sherpa.onnx.OnlineTransducerModelConfig
import java.io.File
import java.util.concurrent.Executors
import kotlin.math.log10
import kotlin.math.max
import kotlin.math.sqrt
/**
* Gearslip's own speech-to-text, so voice works on phones with no speech service (GrapheneOS ships
* none). whisper.cpp runs FUTO's ACFT Whisper model, which are tuned for short phrases: a reply
* of a few seconds comes back in well under a second on a recent phone.
* none). sherpa-onnx runs a streaming model ([SpeechModels]) on the phone: words come out while
* the driver is still talking, and the model itself decides when they've finished. A larger model
* then reads the whole phrase once more for the final words, since a model that waits for the end
* hears names and places far better than one that has to keep up. The phrase is held in memory
* for that and nowhere else, and dropped as soon as it's read.
*
* Whisper needs the whole phrase before it starts, so this records from the car's microphone (the
* phone's when the car has none or sends nothing),
* decides when the driver has stopped talking (a stretch of quiet after speech), then transcribes
* once. Everything runs on one worker thread; callbacks arrive on the main thread.
* Audio comes from the car's microphone, or the phone's when the car has none or sends nothing.
* Everything runs on one worker thread; callbacks arrive on the main thread.
*/
object SpeechEngine {
/** FUTO's ACFT Whisper tiny (Apache-2.0), re-quantized to 5 bits, inside the APK. */
private const val MODEL = "speech/tiny_acft_q5_1.bin"
/** The error for an engine that can't run at all, which callers may answer with a fallback. */
const val NOT_LOADED = "The speech model didn't load."
/** Said when the chosen model hasn't been downloaded yet. */
const val NOT_DOWNLOADED = "Download a voice model in Gearslip's settings on the phone first."
private const val RATE = 16_000
private const val FRAME = 480 // 30 ms
private const val MAX_MS = 15_000L
private const val NO_SPEECH_MS = 6_000L
private const val END_SILENCE_MS = 1_100L
private const val CALIBRATE_MS = 300L
private const val PRE_ROLL = RATE * 3 / 10 // keep 300 ms before the first word
private const val CHUNK = 1_600 // 100 ms
private const val MAX_S = 15f
/** Quiet before the first word that counts as "nobody spoke". */
private const val NO_SPEECH_S = 6f
/** Quiet after the last word that ends the phrase. */
private const val END_SILENCE_S = 1.0f
private const val CUE_VOLUME = 80
private const val IDLE_UNLOAD_MS = 5 * 60_000L
/** How long the car gets to start sending once asked, before the phone's microphone takes over. */
private const val CAR_FIRST_AUDIO_MS = 1_500L
private const val CAR_GAP_MS = 1_000L
@@ -55,27 +66,79 @@ object SpeechEngine {
private const val CUE_TAIL_MS = 250L
private val worker = Executors.newSingleThreadExecutor { Thread(it, "gearslip-speech") }
/**
* The final model takes seconds to load. On [worker], the first press after connecting
* waited behind it before listening at all; here it loads while the driver talks.
*/
private val finalWorker = Executors.newSingleThreadExecutor { Thread(it, "gearslip-speech-final") }
private val finalLock = Object()
private val main = Handler(Looper.getMainLooper())
private var libraryLoaded = false
private var handle = 0L
private var recognizer: OnlineRecognizer? = null
private var loadedModel: SpeechModels.Model? = null
@Volatile private var finalRecognizer: OfflineRecognizer? = null
@Volatile private var loadedFinal: SpeechModels.Model? = null
/**
* The final model holds most of the memory the two models take, so it goes after a few
* minutes without a command. Kroko stays: it's small, and keeps listening instant.
*/
private val idleUnload = Runnable { unloadFinal("after ${IDLE_UNLOAD_MS / 60_000} minutes without use") }
@Volatile private var run = 0
/** Loads the model ahead of time, so the first reply doesn't wait for it. */
/** Loads the model ahead of time, so the first command doesn't wait for it. */
fun warm(context: Context) {
val app = context.applicationContext
worker.execute { ensureLoaded(app) }
SpeechModels.init(app)
// Kroko first, alone: loading both at once slowed it from a fraction of a second to six.
worker.execute {
ensureLoaded(app)
finalWorker.execute { ensureFinalLoaded(app) }
}
}
private fun unloadFinal(why: String) {
finalWorker.execute {
synchronized(finalLock) {
val model = loadedFinal ?: return@execute
finalRecognizer?.release()
finalRecognizer = null
loadedFinal = null
GearslipLog.i("speech: unloaded ${model.label} $why")
}
}
}
/** Drops the loaded models: the driver picked another or deleted one, or the car went away. */
fun unload() {
main.removeCallbacks(idleUnload)
run++
worker.execute {
recognizer?.release()
recognizer = null
loadedModel = null
}
finalWorker.execute {
synchronized(finalLock) {
finalRecognizer?.release()
finalRecognizer = null
loadedFinal = null
}
}
}
/**
* Listens for one phrase. [cue] plays a short tone first, so the driver knows when to talk.
* [onSpeaking] fires at the first word, [onHeard] when the driver stops and the words are being
* worked out; [onResult] gets the text, [onError] a sentence fit to show on the car screen.
* [onPartial] gets the words so far as they're recognized, [onSpeaking] fires at the first
* word, [onHeard] when the driver stops; [onResult] gets the text, [onError] a sentence fit to
* show on the car screen.
*/
fun listen(
context: Context,
cue: Boolean = true,
onSpeaking: () -> Unit = {},
onPartial: (String) -> Unit = {},
onListening: () -> Unit = {},
onHeard: () -> Unit,
onResult: (String) -> Unit,
onError: (String) -> Unit,
@@ -87,28 +150,29 @@ object SpeechEngine {
}
val id = ++run
fun post(block: () -> Unit) { main.post { if (id == run) block() } }
// Brings Parakeet back if it was unloaded, while the driver talks.
main.removeCallbacks(idleUnload)
finalWorker.execute { ensureFinalLoaded(app) }
worker.execute {
if (id != run) return@execute
if (!ensureLoaded(app)) return@execute post { onError(NOT_LOADED) }
if (!SpeechModels.isReady(app, SpeechModels.selected.value)) return@execute post { onError(NOT_DOWNLOADED) }
val r = ensureLoaded(app) ?: return@execute post { onError(NOT_LOADED) }
VoiceFocus.acquire(app)
val audio = try {
val text = try {
if (cue) cue()
if (id != run) return@execute
record(id) { post(onSpeaking) }
post(onListening)
stream(app, r, id, onSpeaking = { post(onSpeaking) }, onPartial = { post { onPartial(it) } }, onStopped = { post(onHeard) })
} catch (t: Throwable) {
GearslipLog.e("speech: recording failed", t)
GearslipLog.e("speech: listening failed", t)
return@execute post { onError("The microphone isn't available right now.") }
} finally {
VoiceFocus.release()
}
main.removeCallbacks(idleUnload)
main.postDelayed(idleUnload, IDLE_UNLOAD_MS)
if (id != run) return@execute
if (audio == null) return@execute post { onError("Didn't catch that.") }
post(onHeard)
val started = SystemClock.uptimeMillis()
val text = clean(transcribe(audio))
GearslipLog.i(
"speech: ${audio.size * 1000L / RATE} ms of audio took ${SystemClock.uptimeMillis() - started} ms",
)
post { if (text.isEmpty()) onError("Didn't catch that.") else onResult(text) }
}
}
@@ -116,6 +180,185 @@ object SpeechEngine {
/** Stops listening; a phrase already being worked out is dropped. */
fun cancel() { run++ }
/**
* Feeds the microphone to the recognizer in 100 ms pieces, reporting the words as they firm
* up, until the model hears the end of the phrase (or nobody speaks, or it runs too long).
*/
private fun stream(
context: Context,
r: OnlineRecognizer,
id: Int,
onSpeaking: () -> Unit,
onPartial: (String) -> Unit,
onStopped: () -> Unit,
): String {
val mic = carMic() ?: phoneMic()
val s = r.createStream()
val frame = ShortArray(CHUNK)
val samples = FloatArray(CHUNK)
// The whole phrase, for the final read; [MAX_S] and a little over is the most it can hold.
val phrase = FloatArray((RATE * (MAX_S + 1)).toInt())
var held = 0
var last = ""
var spoke = false
var audioMs = 0L
var decodeMs = 0L
try {
while (id == run) {
val n = mic.read(frame)
if (n < 0) break
if (n == 0) continue
for (i in 0 until n) samples[i] = frame[i] / 32768f
val keep = minOf(n, phrase.size - held)
System.arraycopy(samples, 0, phrase, held, keep)
held += keep
s.acceptWaveform(if (n == CHUNK) samples else samples.copyOf(n), RATE)
audioMs += n * 1000L / RATE
val started = SystemClock.uptimeMillis()
while (r.isReady(s)) r.decode(s)
decodeMs += SystemClock.uptimeMillis() - started
val text = clean(r.getResult(s).text)
if (text != last) {
last = text
if (text.isNotEmpty() && !spoke) {
spoke = true
onSpeaking()
}
onPartial(text)
}
if (r.isEndpoint(s)) break
}
s.inputFinished()
while (r.isReady(s)) r.decode(s)
last = clean(r.getResult(s).text)
} finally {
mic.close()
s.release()
}
// Under 1.0 keeps up with the driver; over it, the words fall behind the voice.
GearslipLog.i(
"speech: $audioMs ms of audio, decoding took $decodeMs ms " +
"(${"%.2f".format(decodeMs.toFloat() / max(1L, audioMs))} of real time)",
)
if (last.isEmpty() || id != run) { phrase.fill(0f); return last }
// Still loading (just connected, or back after going idle): the live words now beat
// Parakeet's in several seconds.
val wanted = SpeechModels.final.value
if (wanted == null || finalRecognizer == null || loadedFinal != wanted) {
if (wanted != null) GearslipLog.i("speech: ${wanted.label} isn't loaded yet, keeping the live words")
phrase.fill(0f)
return last
}
onStopped()
val started = SystemClock.uptimeMillis()
// Held for the whole read, so a model swap or the idle unload can't free it mid-decode.
val reread = synchronized(finalLock) {
val final = finalRecognizer?.takeIf { loadedFinal == wanted } ?: run { phrase.fill(0f); return last }
final.createStream().run {
try {
acceptWaveform(phrase.copyOf(held), RATE)
final.decode(this)
clean(final.getResult(this).text)
} finally {
release()
}
}
}
phrase.fill(0f)
GearslipLog.i("speech: the final read took ${SystemClock.uptimeMillis() - started} ms")
if (reread.isEmpty()) return last
if (reread != last) onPartial(reread)
return reread
}
/** The model that rereads the phrase, when one is chosen and downloaded; null otherwise. */
private fun ensureFinalLoaded(context: Context): OfflineRecognizer? = synchronized(finalLock) {
val model = SpeechModels.final.value ?: return null
if (finalRecognizer != null && loadedFinal == model) return finalRecognizer
finalRecognizer?.release()
finalRecognizer = null
if (!SpeechModels.isReady(context, model)) return null
val dir = SpeechModels.dir(context, model)
fun path(name: String) = File(dir, name).absolutePath
val started = SystemClock.uptimeMillis()
return runCatching {
OfflineRecognizer(
config = OfflineRecognizerConfig(
featConfig = FeatureConfig(sampleRate = RATE, featureDim = 80),
modelConfig = if (model.modelType == "canary") {
// English in, English out, and no punctuation: clean() strips it anyway.
OfflineModelConfig(
canary = OfflineCanaryModelConfig(path(model.encoder), path(model.decoder), srcLang = "en", tgtLang = "en", usePnc = false),
tokens = path("tokens.txt"),
numThreads = threads(),
)
} else {
OfflineModelConfig(
transducer = OfflineTransducerModelConfig(path(model.encoder), path(model.decoder), path(model.joiner)),
tokens = path("tokens.txt"),
numThreads = threads(),
modelType = model.modelType,
)
},
decodingMethod = "greedy_search",
),
)
}.onSuccess {
finalRecognizer = it
loadedFinal = model
GearslipLog.i("speech: loaded ${model.label} in ${SystemClock.uptimeMillis() - started} ms")
}.onFailure {
GearslipLog.e("speech: couldn't load ${model.label}", it)
}.getOrNull()
}
private fun ensureLoaded(context: Context): OnlineRecognizer? {
val model = SpeechModels.selected.value
if (recognizer != null && loadedModel == model) return recognizer
recognizer?.release()
recognizer = null
if (!SpeechModels.isReady(context, model)) return null
val dir = SpeechModels.dir(context, model)
fun path(name: String) = File(dir, name).absolutePath
val started = SystemClock.uptimeMillis()
return runCatching {
OnlineRecognizer(
config = OnlineRecognizerConfig(
featConfig = FeatureConfig(sampleRate = RATE, featureDim = 80),
modelConfig = OnlineModelConfig(
transducer = OnlineTransducerModelConfig(path(model.encoder), path(model.decoder), path(model.joiner)),
tokens = path("tokens.txt"),
numThreads = threads(),
modelType = model.modelType,
),
endpointConfig = EndpointConfig(
rule1 = EndpointRule(false, NO_SPEECH_S, 0f),
rule2 = EndpointRule(true, END_SILENCE_S, 0f),
rule3 = EndpointRule(false, 0f, MAX_S),
),
enableEndpoint = true,
decodingMethod = "greedy_search",
),
)
}.onSuccess {
recognizer = it
loadedModel = model
GearslipLog.i("speech: loaded ${model.label} in ${SystemClock.uptimeMillis() - started} ms")
}.onFailure {
GearslipLog.e("speech: couldn't load ${model.label}", it)
}.getOrNull()
}
/** The big cores only: more threads than that slow the model down on a phone. */
private fun threads() = Runtime.getRuntime().availableProcessors().let { if (it >= 8) 4 else max(1, it / 2) }
/**
* Models write in upper case, or with punctuation and capitals of their own; commands are
* matched in plain lower case, so only words and apostrophes are kept.
*/
private fun clean(text: String): String =
text.lowercase().replace(Regex("[^\\p{L}\\p{N}' ]"), " ").replace(Regex("\\s+"), " ").trim()
/**
* A short beep on the media stream, so it plays through the car like everything else, then a
* pause long enough for it to finish there before the microphone opens and hears it.
@@ -130,14 +373,9 @@ object SpeechEngine {
}
}
// --- recording --------------------------------------------------------------------------
// --- microphones -----------------------------------------------------------------------
/**
* Records until the driver stops talking. Returns the phrase from just before the first word,
* or null when nobody spoke. The speech threshold follows the noise in the first 300 ms, so a
* loud cabin needs a louder voice than a quiet one to count as talking.
*/
/** Where a phrase is recorded from: [read] fills [frame] and returns how many samples it got. */
/** Where a phrase is heard from: [read] fills [frame] and returns how many samples it got, -1 at the end. */
private interface Mic {
fun read(frame: ShortArray): Int
fun close()
@@ -148,10 +386,11 @@ object SpeechEngine {
val min = AudioRecord.getMinBufferSize(RATE, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT)
val recorder = AudioRecord(
MediaRecorder.AudioSource.VOICE_RECOGNITION, RATE,
AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, max(min, FRAME * 8),
AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, max(min, CHUNK * 4),
)
check(recorder.state == AudioRecord.STATE_INITIALIZED) { "AudioRecord didn't initialize" }
recorder.startRecording()
GearslipLog.i("speech: listening through the phone's microphone")
return object : Mic {
override fun read(frame: ShortArray) = recorder.read(frame, 0, frame.size)
override fun close() {
@@ -163,7 +402,7 @@ object SpeechEngine {
/**
* The car's microphone, which hears the driver better than a phone in a cup holder. The car
* sends audio in its own sized pieces; they're cut to [FRAME]s here. Null when there's no car
* sends audio in its own sized pieces; they're cut to [CHUNK]s here. Null when there's no car
* microphone, or it sends nothing at first, so the phone's is used instead.
*/
private fun carMic(): Mic? {
@@ -190,105 +429,10 @@ object SpeechEngine {
at += n
if (at == chunk.size) { pending = null; at = 0 }
}
// The car stopped sending mid-phrase: end the recording rather than wait forever.
// The car stopped sending mid-phrase: end the phrase rather than wait forever.
return if (filled == 0) -1 else filled
}
override fun close() = car.stop()
}
}
/**
* Records until the driver stops talking. Returns the phrase from just before the first word,
* or null when nobody spoke. The speech threshold follows the noise in the first 300 ms, so a
* loud cabin needs a louder voice than a quiet one to count as talking.
*/
private fun record(id: Int, onSpeaking: () -> Unit): FloatArray? {
val mic = carMic() ?: phoneMic()
val samples = FloatArray((RATE * MAX_MS / 1000).toInt())
var count = 0
val frame = ShortArray(FRAME)
var noise = 0.0
var noiseFrames = 0
var speechStart = -1
var lastSpeech = 0
try {
while (id == run && count + FRAME <= samples.size) {
val n = mic.read(frame)
if (n < 0) break
if (n == 0) continue
var sum = 0.0
for (i in 0 until n) {
val v = frame[i] / 32768f
samples[count + i] = v
sum += v * v
}
count += n
val db = 20 * log10(sqrt(sum / n) + 1e-9)
val ms = count * 1000L / RATE
if (ms <= CALIBRATE_MS) {
noise += db; noiseFrames++
continue
}
val floor = if (noiseFrames > 0) noise / noiseFrames else -60.0
val talking = db > max(floor + 12, -45.0)
if (talking) {
if (speechStart < 0) {
speechStart = max(0, count - n - PRE_ROLL)
onSpeaking()
}
lastSpeech = count
}
if (speechStart < 0 && ms > NO_SPEECH_MS) return null
if (speechStart >= 0 && (count - lastSpeech) * 1000L / RATE > END_SILENCE_MS) break
}
} finally {
mic.close()
}
if (speechStart < 0) return null
return samples.copyOfRange(speechStart, count)
}
// --- the model --------------------------------------------------------------------------
private fun ensureLoaded(context: Context): Boolean {
if (handle != 0L) return true
if (!libraryLoaded) {
runCatching { System.loadLibrary("gearslip_speech") }
.onFailure { GearslipLog.e("speech: native library missing", it); return false }
libraryLoaded = true
}
val started = SystemClock.uptimeMillis()
handle = nativeInitAsset(context.assets, MODEL)
if (handle == 0L) {
GearslipLog.w("speech: couldn't load the model")
return false
}
GearslipLog.i("speech: loaded the model in ${SystemClock.uptimeMillis() - started} ms")
return true
}
private fun transcribe(audio: FloatArray): String {
val bytes = nativeTranscribe(handle, audio, language(), threads())
return String(bytes, Charsets.UTF_8)
}
/** The phone's language, as Whisper names it. Older Java codes differ for a few. */
private fun language(): String = when (val lang = Locale.getDefault().language) {
"iw" -> "he"
"in" -> "id"
"ji" -> "yi"
"" -> "en"
else -> lang
}
/** The big cores only: more threads than that slow Whisper down on a phone. */
private fun threads() = Runtime.getRuntime().availableProcessors().coerceIn(1, 8).let { if (it >= 8) 4 else max(1, it / 2) }
/** Whisper marks silence and noise as "[BLANK_AUDIO]" or "(wind blowing)"; none of it is words. */
private fun clean(text: String): String =
text.replace(Regex("""\[[^\]]*\]|\([^)]*\)|\*[^*]*\*"""), " ").replace(Regex("\\s+"), " ").trim()
private external fun nativeInitAsset(assets: AssetManager, path: String): Long
private external fun nativeTranscribe(handle: Long, samples: FloatArray, language: String, threads: Int): ByteArray
private external fun nativeFree(handle: Long)
}
@@ -0,0 +1,289 @@
package app.seb3thehacker.gearslip.speech
import android.Manifest
import android.app.NotificationChannel
import android.app.NotificationManager
import android.app.PendingIntent
import android.content.Context
import android.content.Intent
import android.content.pm.PackageManager
import androidx.core.app.NotificationCompat
import androidx.core.app.NotificationManagerCompat
import app.seb3thehacker.gearslip.GearslipActivity
import app.seb3thehacker.gearslip.GearslipLog
import java.io.File
import java.net.HttpURLConnection
import java.net.URL
import java.util.concurrent.Executors
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* The speech models Gearslip listens with, in two roles: a streaming one that writes the words
* while the driver talks, and an optional final one that reads the whole phrase again once they
* stop, for a more accurate result. None ships in the APK: each is fetched once and kept in the
* app's own storage. sherpa-onnx's author publishes them twice: as .tar.bz2 archives on GitHub and
* as plain files on Hugging Face. Gearslip takes the plain files: unpacking bzip2 on the phone
* pinned a core and held a download to 150 KB/s on Wi-Fi that does 19 MB/s.
*/
object SpeechModels {
enum class Role { LIVE, FINAL }
enum class Model(
val role: Role,
val id: String,
val label: String,
val detail: String,
val downloadMb: Int,
private val archive: String,
val encoder: String,
val decoder: String,
/** Empty for Canary, which decodes with attention and has no joiner. */
val joiner: String,
val modelType: String = "",
/** Where the unpacked files are, when not under the archive's own name on Hugging Face. */
private val filesAt: String? = null,
) {
KROKO(
Role.LIVE, "kroko-en", "Kroko", "Banafo's Kroko ASR, CC BY-SA 4.0", 68,
"sherpa-onnx-streaming-zipformer-en-kroko-2025-08-06",
"encoder.onnx", "decoder.onnx", "joiner.onnx", modelType = "zipformer2",
),
PARAKEET_SMALL(
Role.FINAL, "parakeet-110m-en", "Parakeet small", "Lighter and quicker, misses more names. NVIDIA NeMo, CC BY 4.0", 131,
"sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000-int8",
"encoder.int8.onnx", "decoder.int8.onnx", "joiner.int8.onnx", modelType = "nemo_transducer",
// The official unpacked copy is empty. This mirror's files are byte for byte the
// official archive's, checked by SHA-256, and pinned to that commit so they can't change.
filesAt = "punitd/sherpa-onnx-nemo-parakeet_tdt_transducer_110m-en-36000-int8/resolve/66a4fa70643dc7ce25c9b38b2f87e1b35ddad33d",
),
CANARY(
Role.FINAL, "canary-180m-en", "Canary", "Recommended. Quick, and hears names and places well. Knows English, Spanish, German and French; Gearslip listens in English for now. NVIDIA NeMo, CC BY 4.0", 198,
"sherpa-onnx-nemo-canary-180m-flash-en-es-de-fr-int8",
"encoder.int8.onnx", "decoder.int8.onnx", "", modelType = "canary",
),
PARAKEET_LARGE(
Role.FINAL, "parakeet-0.6b-en", "Parakeet large", "Hears names best, but slower and three times the size. NVIDIA NeMo, CC BY 4.0", 631,
"sherpa-onnx-nemo-parakeet-tdt-0.6b-v2-int8",
"encoder.int8.onnx", "decoder.int8.onnx", "joiner.int8.onnx", modelType = "nemo_transducer",
),
;
fun fileUrl(name: String) = "https://huggingface.co/${filesAt ?: "csukuangfj/$archive/resolve/main"}/$name"
val files get() = listOf(encoder, decoder, joiner, "tokens.txt").filter { it.isNotEmpty() }
}
/** What setup offers in one tap: Kroko for the live words, Canary for the final ones. */
val RECOMMENDED = listOf(Model.KROKO, Model.CANARY)
fun downloadRecommended(context: Context) = RECOMMENDED.forEach { download(context, it) }
sealed interface State {
data object Missing : State
data class Downloading(val percent: Int) : State
data object Ready : State
data class Failed(val reason: String) : State
}
private const val PREFS = "speech"
private const val KEY_MODEL = "model"
private const val KEY_FINAL = "final_model"
private const val CHANNEL = "voice_download"
private const val NOTIFICATION_ID = 4_312
private val worker = Executors.newSingleThreadExecutor { Thread(it, "gearslip-speech-download") }
private val states = Model.entries.associateWith { MutableStateFlow<State>(State.Missing) }
private val _selected = MutableStateFlow(Model.KROKO)
val selected: StateFlow<Model> = _selected.asStateFlow()
private val _final = MutableStateFlow<Model?>(Model.CANARY)
/** The model that rereads the whole phrase at the end; null to keep the live model's words. */
val final: StateFlow<Model?> = _final.asStateFlow()
@Volatile private var cancelled: Model? = null
fun init(context: Context) {
val id = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getString(KEY_MODEL, null)
_selected.value = Model.entries.firstOrNull { it.id == id && it.role == Role.LIVE } ?: Model.KROKO
// Canary is the default. A driver who never chose but already has Parakeet large keeps it.
val finalId = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).getString(KEY_FINAL, null)
?: if (isReady(context, Model.PARAKEET_LARGE) && !isReady(context, Model.CANARY)) Model.PARAKEET_LARGE.id else Model.CANARY.id
_final.value = Model.entries.firstOrNull { it.id == finalId && it.role == Role.FINAL }
Model.entries.forEach { m -> if (isReady(context, m)) states.getValue(m).value = State.Ready }
// Models an older build offered and this one doesn't: hundreds of MB nobody can use. A
// known model's ".partial" folder is a download that may be running right now; [download]
// clears its own leftovers when it starts.
val known = Model.entries.map { it.id }.toSet()
File(context.filesDir, "speech").listFiles()
?.filter { it.isDirectory && it.name.removeSuffix(".partial") !in known }?.forEach {
GearslipLog.i("speech: removing ${it.name}, no longer offered")
it.deleteRecursively()
}
}
fun state(model: Model): StateFlow<State> = states.getValue(model).asStateFlow()
/**
* The models fetched since the last batch finished, so the percent covers all of them and
* doesn't jump back to 0 when the second one starts.
*/
private val batch = mutableSetOf<Model>()
private val _progress = MutableStateFlow<Int?>(null)
/** Percent through the current downloads, for Home and the notification; null when none run. */
val progress: StateFlow<Int?> = _progress.asStateFlow()
private fun updateProgress(context: Context) {
val percent = synchronized(batch) {
if (batch.none { states.getValue(it).value is State.Downloading }) {
batch.clear()
null
} else {
val done = batch.sumOf { m ->
m.downloadMb * when (val st = states.getValue(m).value) {
is State.Downloading -> st.percent
State.Ready -> 100
else -> 0
}
}
done / batch.sumOf { it.downloadMb }
}
}
if (percent == _progress.value) return
_progress.value = percent
showProgress(context, percent)
}
/** A silent notification with a progress bar, so the download is visible with the app closed. */
private fun showProgress(context: Context, percent: Int?) {
val manager = NotificationManagerCompat.from(context)
if (percent == null) {
manager.cancel(NOTIFICATION_ID)
return
}
if (context.checkSelfPermission(Manifest.permission.POST_NOTIFICATIONS) != PackageManager.PERMISSION_GRANTED) return
context.getSystemService(NotificationManager::class.java).createNotificationChannel(
NotificationChannel(CHANNEL, "Voice download", NotificationManager.IMPORTANCE_LOW),
)
val open = PendingIntent.getActivity(
context, NOTIFICATION_ID,
Intent(context, GearslipActivity::class.java).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK or Intent.FLAG_ACTIVITY_CLEAR_TOP),
PendingIntent.FLAG_IMMUTABLE,
)
val notification = NotificationCompat.Builder(context, CHANNEL)
.setSmallIcon(android.R.drawable.stat_sys_download)
.setContentTitle("Downloading voice")
.setContentText("$percent%")
.setProgress(100, percent, false)
.setOngoing(true)
.setOnlyAlertOnce(true)
.setSilent(true)
.setContentIntent(open)
.build()
runCatching { manager.notify(NOTIFICATION_ID, notification) }
}
fun dir(context: Context, model: Model) = File(File(context.filesDir, "speech"), model.id)
fun isReady(context: Context, model: Model) = model.files.all { File(dir(context, model), it).isFile }
fun select(context: Context, model: Model) {
if (model.role == Role.FINAL) return selectFinal(context, model)
_selected.value = model
context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit().putString(KEY_MODEL, model.id).apply()
SpeechEngine.unload()
}
/** [model] null keeps the live words as they are. */
fun selectFinal(context: Context, model: Model?) {
_final.value = model
context.getSharedPreferences(PREFS, Context.MODE_PRIVATE).edit().putString(KEY_FINAL, model?.id ?: "none").apply()
SpeechEngine.unload()
}
fun delete(context: Context, model: Model) {
cancelled = model
dir(context, model).deleteRecursively()
states.getValue(model).value = State.Missing
updateProgress(context.applicationContext)
if (model == _selected.value || model == _final.value) SpeechEngine.unload()
}
/** Fetches just the files the recognizer reads. */
fun download(context: Context, model: Model) {
val app = context.applicationContext
val state = states.getValue(model)
if (state.value is State.Downloading || state.value == State.Ready) return
state.value = State.Downloading(0)
synchronized(batch) { batch.add(model) }
updateProgress(app)
cancelled = null
worker.execute {
val target = dir(app, model)
val partial = File(target.parentFile, "${model.id}.partial").apply { deleteRecursively(); mkdirs() }
try {
val total = model.downloadMb * 1_048_576L
var done = 0L
fun progress(read: Long) {
if (cancelled == model) throw InterruptedException("cancelled")
state.value = State.Downloading((((done + read) * 100) / total).toInt().coerceIn(0, 99))
updateProgress(app)
}
for (name in model.files) {
val conn = open(model.fileUrl(name))
val length = conn.contentLengthLong
val file = File(partial, name)
CountingStream(conn.inputStream, ::progress).use { input ->
file.outputStream().use { input.copyTo(it, 256 * 1024) }
}
check(length <= 0 || file.length() == length) { "$name arrived cut short" }
done += file.length()
}
check(model.files.all { File(partial, it).isFile }) { "some files were missing" }
target.deleteRecursively()
check(partial.renameTo(target)) { "couldn't store the model" }
GearslipLog.i("speech: downloaded ${model.label}")
state.value = State.Ready
updateProgress(app)
} catch (e: Exception) {
partial.deleteRecursively()
if (cancelled == model) {
state.value = State.Missing
} else {
GearslipLog.w("speech: download of ${model.label} failed: ${e.message}")
state.value = State.Failed(e.message ?: e.javaClass.simpleName)
}
updateProgress(app)
}
}
}
private fun open(url: String): HttpURLConnection =
(URL(url).openConnection() as HttpURLConnection).apply {
connectTimeout = 15_000
readTimeout = 30_000
instanceFollowRedirects = true
}.also { check(it.responseCode == 200) { "the server answered ${it.responseCode}" } }
/** Counts bytes as they pass, reporting about every megabyte. */
private class CountingStream(
private val inner: java.io.InputStream,
private val onProgress: (Long) -> Unit,
) : java.io.InputStream() {
private var read = 0L
private var reported = 0L
override fun read(): Int = inner.read().also { if (it >= 0) count(1) }
override fun read(b: ByteArray, off: Int, len: Int): Int = inner.read(b, off, len).also { if (it > 0) count(it.toLong()) }
override fun close() = inner.close()
private fun count(n: Long) {
read += n
if (read - reported >= 1_048_576) {
reported = read
onProgress(read)
}
}
}
}
@@ -19,17 +19,35 @@ import app.seb3thehacker.gearslip.GearslipLog
object VoiceFocus {
private const val RELEASE_DELAY_MS = 1_500L
private const val AFTER_RELEASE_MS = 300L
private val main = Handler(Looper.getMainLooper())
private var request: AudioFocusRequest? = null
private var audio: AudioManager? = null
private var holders = 0
private val afterRelease = mutableListOf<() -> Unit>()
private val releaseNow = Runnable {
val r = request ?: return@Runnable
audio?.abandonAudioFocusRequest(r)
request = null
GearslipLog.i("voice: gave audio focus back")
request?.let { r ->
audio?.abandonAudioFocusRequest(r)
request = null
GearslipLog.i("voice: gave audio focus back")
}
val pending = afterRelease.toList()
afterRelease.clear()
// Just after the music app has had its focus back and acted on it.
if (pending.isNotEmpty()) main.postDelayed({ pending.forEach { it() } }, AFTER_RELEASE_MS)
}
/**
* Runs [action] once the music has its focus back, or now if Gearslip isn't holding it.
* Metrolist takes a pause sent while it's held as nothing and resumes when focus returns,
* so "pause" sends it again from here.
*/
fun afterRelease(action: () -> Unit) {
main.post {
if (request == null && holders == 0) action() else afterRelease += action
}
}
/** Pauses the music until a matching [release]. Safe from any thread. */
@@ -32,6 +32,8 @@ object UsageStats {
val android: Boolean = true,
val phone: Boolean = true,
val cars: Boolean = true,
/** How each car session went: how far it got, how and when it ended. Off until ticked. */
val failures: Boolean = false,
)
private const val PREFS = "usage_stats"
@@ -49,14 +51,18 @@ object UsageStats {
/** The daily background job of earlier builds, cancelled wherever it's still scheduled. */
private const val OLD_JOB_ID = 4_207
private const val MAX_CARS = 10
/** Each session for the "Where connections fail" share, waiting for the next note. */
private const val KEY_SESSIONS = "sessions"
private const val KEY_LAST_SESSIONS = "last_sessions"
private const val MAX_SESSIONS = 20
private fun prefs(context: Context) = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
/**
* Dev builds show every switch and screen, so they can be tested, but never send anything:
* testing mustn't muddy the numbers.
* Debug builds send too, marked "debug". The server keeps those notes apart, so testing never
* muddies the numbers.
*/
val sends: Boolean get() = !BuildConfig.DEBUG && BuildConfig.STATS_URL.isNotBlank()
val sends: Boolean get() = BuildConfig.STATS_URL.isNotBlank()
fun enabled(context: Context): Boolean = prefs(context).getBoolean(KEY_ENABLED, false)
@@ -69,20 +75,30 @@ object UsageStats {
android = p.getBoolean("share_android", true),
phone = p.getBoolean("share_phone", true),
cars = p.getBoolean("share_cars", true),
failures = p.getBoolean("share_failures", false),
)
}
fun set(context: Context, enabled: Boolean, choices: Choices = choices(context)) {
val wasEnabled = enabled(context)
prefs(context).edit()
.putBoolean(KEY_ENABLED, enabled)
.putBoolean(KEY_ASKED, true)
.putBoolean("share_android", choices.android)
.putBoolean("share_phone", choices.phone)
.putBoolean("share_cars", choices.cars)
.putBoolean("share_failures", choices.failures)
// The server drops details from a box once it's unticked, so the next note sends
// them again rather than assuming the server still has them.
.remove(KEY_SENT_DETAILS)
.apply()
// Turning it on sends nothing yet: the driver may still be ticking boxes. The first note
// waits for [choicesClosed].
if (!enabled) prefs(context).edit().remove(KEY_CARS).remove(KEY_LAST_CARS).remove(KEY_SENT_DAY).apply()
if (!enabled) prefs(context).edit().remove(KEY_CARS).remove(KEY_LAST_CARS).remove(KEY_SENT_DAY)
.remove(KEY_SESSIONS).remove(KEY_LAST_SESSIONS).apply()
if (!choices.failures) prefs(context).edit().remove(KEY_SESSIONS).remove(KEY_LAST_SESSIONS).apply()
// Turned on anywhere - Settings, Home, or the ask itself - answers the one-time ask.
if (enabled && !wasEnabled) WorkedPrompt.onSharingOn(context)
}
/** The driver left the screen with the usage note choices: send today's note if it's due. */
@@ -90,10 +106,15 @@ object UsageStats {
if (prefs(context).getString(KEY_SENT_DAY, null) != LocalDate.now().toString()) sendSoon(context)
}
/** Gearslip was opened: send today's note unless one already went out. */
/**
* Gearslip was opened: send today's note unless one already went out, or sooner if a session
* is still waiting, like one that ended in a crash before its note could go.
*/
fun onAppOpened(context: Context) {
context.getSystemService(JobScheduler::class.java)?.cancel(OLD_JOB_ID)
if (prefs(context).getString(KEY_SENT_DAY, null) != LocalDate.now().toString()) sendSoon(context)
val p = prefs(context)
val waiting = p.getString(KEY_CARS, "[]") != "[]" || p.getString(KEY_SESSIONS, "[]") != "[]"
if (waiting || p.getString(KEY_SENT_DAY, null) != LocalDate.now().toString()) sendSoon(context)
}
private val sending = AtomicBoolean(false)
@@ -134,13 +155,40 @@ object UsageStats {
outcome: String,
screen: String = "",
dpi: Int = 0,
session: JSONObject? = null,
link: String = "",
audio: String = "",
) {
if (!enabled(context)) return
val p = prefs(context)
synchronized(carsLock) { addCar(p, info, protocol, outcome, screen, dpi) }
synchronized(carsLock) {
addCar(p, info, protocol, outcome, screen, dpi, link, audio)
if (session != null && choices(context).failures) addSession(p, info, protocol, session, link, audio)
}
sendSoon(context)
}
private fun addSession(
p: android.content.SharedPreferences,
info: ServiceDiscovery.HeadUnitInfo?,
protocol: String,
session: JSONObject,
link: String,
audio: String,
) {
val sessions = runCatching { JSONArray(p.getString(KEY_SESSIONS, "[]")) }.getOrDefault(JSONArray())
if (sessions.length() >= MAX_SESSIONS) return
// The same car fields as the cars list, so the dashboard can line the two up.
session.put("name", info?.headUnitName.orEmpty())
.put("car", info?.carModel.orEmpty())
.put("year", info?.carYear.orEmpty())
.put("protocol", protocol)
if (link.isNotEmpty()) session.put("link", link).put("audio", audio)
sessions.put(session)
// Written at once: after a crash the app is about to die, and a deferred write would be lost.
p.edit().putString(KEY_SESSIONS, sessions.toString()).commit()
}
private fun addCar(
p: android.content.SharedPreferences,
info: ServiceDiscovery.HeadUnitInfo?,
@@ -148,6 +196,8 @@ object UsageStats {
outcome: String,
screen: String,
dpi: Int,
link: String,
audio: String,
) {
val cars = runCatching { JSONArray(p.getString(KEY_CARS, "[]")) }.getOrDefault(JSONArray())
val car = JSONObject()
@@ -159,18 +209,45 @@ object UsageStats {
.put("protocol", protocol)
.put("result", outcome)
if (screen.isNotEmpty()) car.put("screen", screen).put("dpi", dpi)
// USB or wireless, and whether music went over USB or Bluetooth: settings, not the person.
if (link.isNotEmpty()) car.put("link", link).put("audio", audio)
// The same car and outcome twice in a day is one fact, not two.
val seen = (0 until cars.length()).any { cars.optJSONObject(it)?.toString() == car.toString() }
if (!seen && cars.length() < MAX_CARS) cars.put(car)
p.edit().putString(KEY_CARS, cars.toString()).apply()
p.edit().putString(KEY_CARS, cars.toString()).commit()
}
// GrapheneOS reports the same build fields as stock Pixel firmware; its own apps give it away.
private fun isGrapheneOs(context: Context) =
runCatching { context.packageManager.getPackageInfo("app.grapheneos.setupwizard", 0) }.isSuccess
/**
* Which Android this is: a privacy OS by name, or "Stock" with the phone's brand.
*
* GrapheneOS reports the same build fields as stock Pixel firmware, so its own setup app gives
* it away; /e/OS is told apart from the LineageOS it's built on by its app store. The rest set
* build properties of their own. The iodéOS and DivestOS checks are best guesses from their
* build names, not confirmed on a real phone.
*/
fun osName(context: Context): String {
fun installed(pkg: String) = runCatching { context.packageManager.getPackageInfo(pkg, 0) }.isSuccess
val names = listOf("ro.build.flavor", "ro.build.display.id", "ro.lineage.version")
.joinToString(" ") { prop(it) } + " " + Build.DISPLAY + " " + Build.FINGERPRINT
return when {
installed("app.grapheneos.setupwizard") -> "GrapheneOS"
prop("ro.calyxos.version").isNotEmpty() -> "CalyxOS"
installed("foundation.e.apps") -> "/e/OS"
names.contains("iode", ignoreCase = true) -> "iodéOS"
names.contains("divest", ignoreCase = true) -> "DivestOS"
prop("ro.lineage.version").isNotEmpty() -> "LineageOS"
else -> "Stock (${Build.BRAND.replaceFirstChar { it.uppercase() }})"
}
}
/** A system property, or "" when it isn't set or can't be read. */
private fun prop(name: String): String = runCatching {
Class.forName("android.os.SystemProperties").getMethod("get", String::class.java)
.invoke(null, name) as String
}.getOrDefault("")
private fun details(context: Context, c: Choices) = JSONObject().apply {
if (c.android) put("android", Build.VERSION.RELEASE).put("os", if (isGrapheneOs(context)) "GrapheneOS" else "other")
if (c.android) put("android", Build.VERSION.RELEASE).put("os", osName(context))
if (c.phone) put("phone", "${Build.MANUFACTURER} ${Build.MODEL}")
}
@@ -185,7 +262,15 @@ object UsageStats {
.put("id", installId(context))
.put("version", BuildConfig.VERSION_NAME)
.put("code", BuildConfig.VERSION_CODE)
.put("build", if (BuildConfig.DEBUG) "dev" else "release")
.put("build", if (BuildConfig.DEBUG) "debug" else "release")
// Which boxes are ticked, so the server knows what this phone shares and can drop
// anything it kept from a box since unticked.
.put("shares", JSONArray().apply {
if (c.android) put("android")
if (c.phone) put("phone")
if (c.cars) put("cars")
if (c.failures) put("sessions")
})
// Phone details change rarely, so they ride along only when they differ from what the
// server last got, or once a month in case it lost them. Unticked ones are left out, and
// the server keeps the last value it had.
@@ -203,9 +288,35 @@ object UsageStats {
}
note.put("cars", cars)
}
if (c.failures) {
var sessions = runCatching { JSONArray(p.getString(KEY_SESSIONS, "[]")) }.getOrDefault(JSONArray())
if (everything && sessions.length() == 0) {
sessions = runCatching { JSONArray(p.getString(KEY_LAST_SESSIONS, "[]")) }.getOrDefault(JSONArray())
}
note.put("sessions", sessions)
}
return note
}
/**
* A made-up session, for "What gets sent" to show before the first real one: a 25-minute
* drive that ended with the car switched off.
*/
val exampleSession: JSONObject
get() = JSONObject()
.put("stage", "streaming")
.put("ended", "byebye")
.put("code", "bye 1")
.put("failed_at", 1500)
.put("seconds", 1500)
.put("cert", "downloaded")
.put("name", "Head unit name")
.put("car", "Car model")
.put("year", "2021")
.put("protocol", "1.5")
.put("link", "usb")
.put("audio", "bluetooth")
/** Sends the note now; false when it didn't get through. */
private fun send(context: Context): Boolean {
if (!sends || !enabled(context)) return true
@@ -213,6 +324,7 @@ object UsageStats {
val p = prefs(context)
val note = synchronized(carsLock) { preview(context) }
val sentCars = note.optJSONArray("cars")?.toString()
val sentSessions = note.optJSONArray("sessions")?.toString()
val sentDetails = if (note.has("android") || note.has("phone")) details(context, choices(context)).toString() else null
val body = note.toString().toByteArray()
return runCatching {
@@ -234,6 +346,16 @@ object UsageStats {
if (sentDetails != null) edit.putString(KEY_SENT_DETAILS, sentDetails).putString(KEY_DETAILS_DAY, LocalDate.now().toString())
if (sentCars != null && sentCars != "[]") edit.putString(KEY_LAST_CARS, sentCars)
if (sentCars == null || p.getString(KEY_CARS, "[]") == sentCars) edit.remove(KEY_CARS)
if (sentSessions != null && sentSessions != "[]") edit.putString(KEY_LAST_SESSIONS, sentSessions)
if (sentSessions == null || p.getString(KEY_SESSIONS, "[]") == sentSessions) {
edit.remove(KEY_SESSIONS)
} else if (sentSessions != "[]") {
// A session ended mid-send: keep only the ones this note didn't carry.
val now = JSONArray(p.getString(KEY_SESSIONS, "[]"))
val sent = JSONArray(sentSessions).length()
val left = JSONArray().also { for (i in sent until now.length()) it.put(now.get(i)) }
edit.putString(KEY_SESSIONS, left.toString())
}
edit.apply()
}
GearslipLog.i("stats: note sent")
@@ -0,0 +1,115 @@
package app.seb3thehacker.gearslip.stats
import android.content.Context
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.ServiceDiscovery
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import org.json.JSONObject
/**
* The one-time "Gearslip worked with your car. Share that?" ask, for drivers who never answered
* the usage notes question in setup, either way. It comes after the first drive that worked, when a driver is happiest to help, as
* a card on Home. Answering it clears it, and it never comes back.
*
* Saying yes turns on the same usage notes switch as Settings, and that drive's car goes in the
* first note. Until the driver answers, the car waits here on the phone and is sent nowhere.
*/
object WorkedPrompt {
private const val PREFS = "worked_prompt"
private const val KEY_DONE = "done"
private const val KEY_CAR = "car"
private fun prefs(context: Context) = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
private var loaded = false
private val flow = MutableStateFlow<String?>(null)
/** The car's name while the ask is waiting for an answer, otherwise null. */
fun pending(context: Context): StateFlow<String?> {
if (!loaded) {
loaded = true
flow.value = saved(context)?.takeUnless { UsageStats.asked(context) }?.let(::carName)
}
return flow
}
/** A drive that reached the car's screen ended cleanly. Asks, if it's never asked before. */
fun onDriveWorked(context: Context, info: ServiceDiscovery.HeadUnitInfo?, protocol: String, screen: String, dpi: Int) {
val p = prefs(context)
// Asked once only: a driver who answered in setup already said yes or no.
if (UsageStats.asked(context) || p.getBoolean(KEY_DONE, false) || p.contains(KEY_CAR)) return
val car = JSONObject()
.put("name", info?.headUnitName.orEmpty())
.put("car", info?.carModel.orEmpty())
.put("year", info?.carYear.orEmpty())
.put("make", info?.headUnitMake.orEmpty())
.put("model", info?.headUnitModel.orEmpty())
.put("protocol", protocol)
.put("screen", screen)
.put("dpi", dpi)
p.edit().putString(KEY_CAR, car.toString()).apply()
loaded = true
flow.value = carName(car)
}
/** True once a driver turns sharing on from the ask, until Home has thrown its confetti. */
private val celebrateFlow = MutableStateFlow(false)
val celebrate: StateFlow<Boolean> = celebrateFlow
fun celebrated() { celebrateFlow.value = false }
/** The driver answered on Home. Either way it's never asked again. */
fun answer(context: Context, share: Boolean) {
if (share) {
// Turning sharing on lands in [onSharingOn], which clears the ask.
UsageStats.set(context, true)
return
}
clear(context)
GearslipLog.i("worked prompt: not now")
}
/**
* Usage notes were just turned on, from the ask or anywhere else. With the ask still waiting,
* that answers it, and the car it was about goes in the first note.
*/
internal fun onSharingOn(context: Context) {
val car = saved(context) ?: return
clear(context)
celebrateFlow.value = true
GearslipLog.i("worked prompt: usage notes turned on")
val info = ServiceDiscovery.HeadUnitInfo(
car.optString("name"), car.optString("car"), car.optString("year"),
car.optString("make"), car.optString("model"),
)
UsageStats.recordSession(
context, info, car.optString("protocol"), "connected",
car.optString("screen"), car.optInt("dpi"),
)
}
private fun clear(context: Context) {
prefs(context).edit().remove(KEY_CAR).putBoolean(KEY_DONE, true).apply()
loaded = true
flow.value = null
}
private fun saved(context: Context): JSONObject? =
prefs(context).getString(KEY_CAR, null)?.let { runCatching { JSONObject(it) }.getOrNull() }
/** "your 2019 Clio", "your Renault Media Nav", or "your car" when it didn't say. */
private fun carName(car: JSONObject): String {
val model = car.optString("car").trim()
val year = car.optString("year").trim().takeIf { it.isNotEmpty() && it != "0" }
val unit = "${car.optString("make")} ${car.optString("model")}".trim()
return when {
model.isNotEmpty() -> "your " + listOfNotNull(year, model).joinToString(" ")
unit.isNotEmpty() -> "your $unit"
else -> "your car"
}
}
const val TITLE_START = "Gearslip worked with "
}
@@ -0,0 +1,189 @@
package app.seb3thehacker.gearslip.ui
import android.content.ClipData
import android.content.ClipboardManager
import android.content.Context
import android.content.Intent
import android.net.Uri
import android.widget.Toast
import androidx.compose.foundation.clickable
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.automirrored.filled.Send
import androidx.compose.material.icons.filled.Build
import androidx.compose.material.icons.filled.Favorite
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.text.selection.SelectionContainer
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.unit.dp
import androidx.compose.ui.platform.LocalContext
/** Where Gearslip lives outside the app: support, the chat room, and the code. */
internal object GearslipLinks {
const val BUY_ME_A_COFFEE = "https://buymeacoffee.com/seb3thehacker"
const val LIBERAPAY = "https://liberapay.com/Seb3thehacker"
/** A card payment page for giving once, any amount. Its row stays hidden while this is empty. */
const val PAYMENT_LINK = ""
/** Gearslip's Monero address. Its row stays hidden while this is empty. */
const val MONERO = "44gTVxQJY3cirBJ41J5i1bPUVnSkTMhn9esSp1kg9MHwJ3TUz8tkkigUTKwiok5zA5RZvNxyM4pqkAfPs79pPLfaNG21pCn"
const val MATRIX = "https://matrix.to/#/!mf0_mgBd5KqKpc9FfX_wHp1jFCcwAtPrndIn_oGIyMk?via=matrix.org"
const val SOURCE = "https://github.com/twinksoftware/gearslip"
const val CARS = "$SOURCE/blob/main/CARS.md"
fun open(context: Context, url: String) {
runCatching {
context.startActivity(Intent(Intent.ACTION_VIEW, Uri.parse(url)).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
}
}
/** Opens a Monero wallet app on the address; says so when none is installed. */
fun openMoneroWallet(context: Context) {
runCatching {
context.startActivity(Intent(Intent.ACTION_VIEW, Uri.parse("monero:$MONERO")).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
}.onFailure {
Toast.makeText(context, "No Monero wallet app found. Copy the address instead.", Toast.LENGTH_LONG).show()
}
}
fun copyMonero(context: Context) {
context.getSystemService(ClipboardManager::class.java)
?.setPrimaryClip(ClipData.newPlainText("Monero address", MONERO))
Toast.makeText(context, "Monero address copied", Toast.LENGTH_SHORT).show()
}
}
/** The ways to support Gearslip: Liberapay first, Monero last, card once it's set up. */
@Composable
internal fun DonateRows() {
val context = LocalContext.current
val arrow = @Composable {
Icon(
Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
SettingsRow(
"Liberapay",
Modifier.clickable { GearslipLinks.open(context, GearslipLinks.LIBERAPAY) },
subtitle = "Give weekly or monthly",
icon = Icons.Filled.Favorite,
trailing = arrow,
)
SettingsDivider()
SettingsRow(
"Buy Me a Coffee",
Modifier.clickable { GearslipLinks.open(context, GearslipLinks.BUY_ME_A_COFFEE) },
subtitle = "Give once or monthly, by card. Buy Me a Coffee keeps 5%",
icon = Icons.Filled.Favorite,
trailing = arrow,
)
if (GearslipLinks.PAYMENT_LINK.isNotEmpty()) {
SettingsDivider()
SettingsRow(
"Give once",
Modifier.clickable { GearslipLinks.open(context, GearslipLinks.PAYMENT_LINK) },
subtitle = "Any amount, by card",
icon = Icons.Filled.Favorite,
trailing = arrow,
)
}
if (GearslipLinks.MONERO.isNotEmpty()) {
var moneroOpen by remember { mutableStateOf(false) }
SettingsDivider()
SettingsRow(
"Monero",
Modifier.clickable { moneroOpen = true },
subtitle = "Open your wallet or copy the address",
icon = Icons.Filled.Favorite,
trailing = arrow,
)
if (moneroOpen) MoneroDialog { moneroOpen = false }
}
}
/** The Monero address in full, with ways to pay it from this phone or another device. */
@Composable
internal fun MoneroDialog(onDismiss: () -> Unit) {
val context = LocalContext.current
AlertDialog(
onDismissRequest = onDismiss,
icon = { Icon(Icons.Filled.Favorite, contentDescription = null) },
title = { Text("Support with Monero") },
text = {
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Text("Send any amount to this address from your Monero wallet.")
SelectionContainer {
Text(
GearslipLinks.MONERO,
style = MaterialTheme.typography.bodySmall.copy(fontFamily = FontFamily.Monospace),
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
},
confirmButton = {
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Button(onClick = { GearslipLinks.openMoneroWallet(context) }, Modifier.fillMaxWidth()) {
Text("Open in wallet")
}
OutlinedButton(onClick = { GearslipLinks.copyMonero(context) }, Modifier.fillMaxWidth()) {
Text("Copy address")
}
TextButton(onClick = onDismiss, Modifier.fillMaxWidth()) {
Text("Close")
}
}
},
)
}
/** The About section's links: the chat room, the source code, and credits. */
@Composable
internal fun AboutLinkRows() {
val context = LocalContext.current
val arrow = @Composable {
Icon(
Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
SettingsRow(
"Join the chat on Matrix",
Modifier.clickable { GearslipLinks.open(context, GearslipLinks.MATRIX) },
subtitle = "Talk with other drivers about cars, apps, and Gearslip",
icon = Icons.AutoMirrored.Filled.Send,
trailing = arrow,
)
SettingsDivider()
SettingsRow(
"Source code",
Modifier.clickable { GearslipLinks.open(context, GearslipLinks.SOURCE) },
subtitle = "Read or change Gearslip's code under the AGPL",
icon = Icons.Filled.Build,
trailing = arrow,
)
SettingsDivider()
SettingsRow(
"Voice credits",
subtitle = "Speech recognition by sherpa-onnx (Apache 2.0), with Kroko ASR by Banafo " +
"(CC BY-SA 4.0), and Parakeet and Canary by NVIDIA (CC BY 4.0)",
icon = Icons.Filled.Favorite,
)
}
@@ -0,0 +1,71 @@
package app.seb3thehacker.gearslip.ui
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.unit.dp
import app.seb3thehacker.gearslip.BuildConfig
/**
* Shown once, the first time the driver turns down Android's screen-sharing dialog: that dialog
* is the price of sending music over USB, and most people who refuse it would rather not see it
* again. Offers the two ways out: Bluetooth, or the one-time adb command.
*/
@Composable
fun AudioChoiceDialog(onUseBluetooth: () -> Unit, onKeepAsking: () -> Unit) {
var showCommand by remember { mutableStateOf(false) }
AlertDialog(
onDismissRequest = onKeepAsking,
title = { Text("Play car audio over Bluetooth?") },
text = {
Column(verticalArrangement = Arrangement.spacedBy(12.dp)) {
Text(
"Android treats music sent over USB like sharing your screen, so it asks every " +
"drive. Over Bluetooth it never asks. You can change this later in Settings.",
)
if (showCommand) {
Text(
"Or keep USB and stop the question for good: plug the phone into a " +
"computer with USB debugging on, and run this once.",
style = MaterialTheme.typography.bodyMedium,
)
CopyableCommand("adb shell appops set ${BuildConfig.APPLICATION_ID} PROJECT_MEDIA allow")
} else {
TextButton(onClick = { showCommand = true }) { Text("Never ask, using a computer") }
}
}
},
confirmButton = { TextButton(onClick = onUseBluetooth) { Text("Use Bluetooth") } },
dismissButton = { TextButton(onClick = onKeepAsking) { Text("Keep asking") } },
)
}
/**
* Android lets an app capture other apps' audio only while it holds the microphone permission.
* Once the driver has refused that twice, Android stops asking, and music silently stays on the
* phone; this says why and where to turn it back on.
*/
@Composable
fun MicBlockedDialog(onOpenSettings: () -> Unit, onDismiss: () -> Unit) {
AlertDialog(
onDismissRequest = onDismiss,
title = { Text("Car audio needs the microphone permission") },
text = {
Text(
"Android lets an app send other apps' sound to the car only if it has the " +
"microphone permission. It's turned off for Gearslip, so music plays on the " +
"phone. Turn it on under Permissions, then Microphone.",
)
},
confirmButton = { TextButton(onClick = onOpenSettings) { Text("Open app settings") } },
dismissButton = { TextButton(onClick = onDismiss) { Text("Not now") } },
)
}
@@ -60,7 +60,6 @@ fun CertSetupScreen(onDone: () -> Unit) {
val scope = rememberCoroutineScope()
var downloadState by remember { mutableStateOf<DownloadState>(DownloadState.Idle) }
var showImport by remember { mutableStateOf(false) }
var pendingUri by remember { mutableStateOf<Uri?>(null) }
var manualProgress by remember { mutableStateOf(false) }
@@ -91,7 +90,7 @@ fun CertSetupScreen(onDone: () -> Unit) {
if (installed) {
"Cars accept only Google-issued certificates, and you have one loaded."
} else {
"Cars accept only Google-issued certificates. Download one to connect."
"Cars accept only Google-issued certificates. Gearslip doesn't include one, so choose where yours comes from."
},
style = MaterialTheme.typography.bodyLarge,
textAlign = TextAlign.Center,
@@ -131,10 +130,11 @@ fun CertSetupScreen(onDone: () -> Unit) {
},
modifier = Modifier.fillMaxWidth().height(56.dp),
) {
Text("Download certificate")
Text("Download from aasdk")
}
Text(
"Works with most cars built before 2020. Newer cars reject it.",
"The open-source aasdk project publishes this certificate at github.com/opencardev/aasdk. " +
"Most cars accept it; some from around 2020 on don't.",
style = MaterialTheme.typography.bodySmall,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurfaceVariant,
@@ -177,16 +177,18 @@ fun CertSetupScreen(onDone: () -> Unit) {
Spacer(Modifier.height(16.dp))
if (showImport) {
OutlinedButton(
onClick = { importLauncher.launch(arrayOf("*/*")) },
modifier = Modifier.fillMaxWidth().height(52.dp),
) { Text("Import your own .p12 cert") }
Spacer(Modifier.height(16.dp))
} else {
TextButton(onClick = { showImport = true }) { Text("Import your own certificate instead") }
Spacer(Modifier.height(8.dp))
}
OutlinedButton(
onClick = { importLauncher.launch(arrayOf("*/*")) },
modifier = Modifier.fillMaxWidth().height(52.dp),
) { Text("Import my own file") }
Text(
"A .p12 file with its private key, from wherever you choose.",
style = MaterialTheme.typography.bodySmall,
textAlign = TextAlign.Center,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(top = 8.dp),
)
Spacer(Modifier.height(16.dp))
OutlinedButton(
onClick = {
@@ -14,8 +14,17 @@ data class CertSummary(
val validUntil: String?,
) {
companion object {
/**
* The last [read], so Settings draws the certificate row at once instead of a second
* later, which shifted everything under it. Null until the first read.
*/
@Volatile var cached: CertSummary? = null
private set
/** Blocking (parses a key store): call from a background dispatcher. */
fun read(context: Context): CertSummary {
fun read(context: Context): CertSummary = readNow(context).also { cached = it }
private fun readNow(context: Context): CertSummary {
val identity = CertProvider.loadSupplied(context)
?: return CertSummary(
CertProvider.Kind.SELF_SIGNED,
@@ -0,0 +1,88 @@
package app.seb3thehacker.gearslip.ui
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableFloatStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.runtime.withFrameNanos
import androidx.compose.ui.Modifier
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.rotate
import kotlin.math.sin
import kotlin.random.Random
private class Piece(
val x: Float, // across the screen, 0..1
val delay: Float, // seconds before it starts falling
val speed: Float, // screen heights per second
val sway: Float, // how far it drifts side to side, in screen widths
val swaySpeed: Float,
val spin: Float, // degrees per second
val width: Float, // dp
val height: Float,
val color: Color,
)
/**
* Confetti falling from the top of the screen once, for about three seconds. Draws over whatever
* is under it and takes no touches; [onDone] fires when the last piece has left the screen.
*/
@Composable
internal fun ConfettiFall(onDone: () -> Unit, modifier: Modifier = Modifier) {
val scheme = MaterialTheme.colorScheme
val colors = listOf(
scheme.primary, scheme.tertiary, scheme.secondary,
Color(0xFFFFC107), Color(0xFFE91E63), Color(0xFF4CAF50), Color(0xFF2196F3),
)
val pieces = remember {
List(PIECES) {
Piece(
x = Random.nextFloat(),
delay = Random.nextFloat() * 0.9f,
speed = 0.35f + Random.nextFloat() * 0.35f,
sway = 0.01f + Random.nextFloat() * 0.03f,
swaySpeed = 2f + Random.nextFloat() * 4f,
spin = (Random.nextFloat() - 0.5f) * 720f,
width = 4f + Random.nextFloat() * 4f,
height = 8f + Random.nextFloat() * 6f,
color = colors[Random.nextInt(colors.size)],
)
}
}
var t by remember { mutableFloatStateOf(0f) }
LaunchedEffect(Unit) {
val start = withFrameNanos { it }
// The slowest piece, starting last, clears the screen by this time.
val end = pieces.maxOf { it.delay + 1.1f / it.speed }
while (t < end) {
withFrameNanos { t = (it - start) / 1e9f }
}
onDone()
}
Canvas(modifier.fillMaxSize()) {
val density = density
for (p in pieces) {
val age = t - p.delay
if (age < 0f) continue
val y = (age * p.speed - 0.05f) * size.height
if (y > size.height + 40f) continue
val x = (p.x + sin(age * p.swaySpeed) * p.sway) * size.width
rotate(age * p.spin, pivot = Offset(x, y)) {
drawRect(
p.color,
topLeft = Offset(x - p.width * density / 2, y - p.height * density / 2),
size = Size(p.width * density, p.height * density),
)
}
}
}
}
private const val PIECES = 140
@@ -0,0 +1,132 @@
package app.seb3thehacker.gearslip.ui
import android.content.Context
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Favorite
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.Button
import androidx.compose.material3.Icon
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import app.seb3thehacker.gearslip.GearslipLog
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* The donation ask on Home, after a drive that worked. A driver sees it first after their third
* such drive, so Gearslip has earned it, and then at most once every three months. Tapping either
* donate button ends it for good.
*/
object DonatePrompt {
private const val PREFS = "donate_prompt"
private const val KEY_DRIVES = "drives"
private const val KEY_LAST = "last_shown"
private const val KEY_DONATED = "donated"
private const val KEY_WAITING = "waiting"
private const val FIRST_AFTER = 3
private const val GAP_MS = 90L * 24 * 60 * 60 * 1000
private fun prefs(context: Context) = context.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
private var loaded = false
private val flow = MutableStateFlow(false)
/** True while the ask is waiting for the driver to open Home. */
fun pending(context: Context): StateFlow<Boolean> {
if (!loaded) {
loaded = true
flow.value = prefs(context).getBoolean(KEY_WAITING, false)
}
return flow
}
/** A drive that reached the car's screen ended cleanly. Asks if it's time. */
fun onDriveWorked(context: Context) {
val p = prefs(context)
if (p.getBoolean(KEY_DONATED, false)) return
val drives = p.getInt(KEY_DRIVES, 0) + 1
val due = drives >= FIRST_AFTER && System.currentTimeMillis() - p.getLong(KEY_LAST, 0L) > GAP_MS
p.edit().putInt(KEY_DRIVES, drives).apply {
if (due) putBoolean(KEY_WAITING, true)
}.apply()
if (due) {
loaded = true
flow.value = true
}
}
/** The driver closed the ask; [donated] if they picked a way to give. */
fun answer(context: Context, donated: Boolean) {
prefs(context).edit()
.putBoolean(KEY_WAITING, false)
.putLong(KEY_LAST, System.currentTimeMillis())
.apply { if (donated) putBoolean(KEY_DONATED, true) }
.apply()
loaded = true
flow.value = false
GearslipLog.i(if (donated) "donate prompt: opened a donation link" else "donate prompt: not now")
}
}
/** Shows the donation ask over Home when one is waiting and the car isn't connected. */
@Composable
internal fun DonateDialog(live: Boolean) {
val context = LocalContext.current
val waiting by remember { DonatePrompt.pending(context) }.collectAsStateWithLifecycle()
var monero by remember { mutableStateOf(false) }
if (!waiting || live) return
if (monero) {
MoneroDialog { DonatePrompt.answer(context, donated = true) }
return
}
fun give(url: String) {
DonatePrompt.answer(context, donated = true)
GearslipLinks.open(context, url)
}
AlertDialog(
onDismissRequest = { DonatePrompt.answer(context, donated = false) },
icon = { Icon(Icons.Filled.Favorite, contentDescription = null) },
title = { Text("Glad that drive worked") },
text = {
Text(
"Gearslip is free, with no ads and no tracking. If it gets you on the road, your " +
"support keeps it going.",
)
},
confirmButton = {
Column(Modifier.fillMaxWidth(), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Button(onClick = { give(GearslipLinks.LIBERAPAY) }, Modifier.fillMaxWidth()) {
Text("Support on Liberapay")
}
OutlinedButton(onClick = { give(GearslipLinks.BUY_ME_A_COFFEE) }, Modifier.fillMaxWidth()) {
Text("Buy me a coffee")
}
if (GearslipLinks.PAYMENT_LINK.isNotEmpty()) {
OutlinedButton(onClick = { give(GearslipLinks.PAYMENT_LINK) }, Modifier.fillMaxWidth()) {
Text("Give once by card")
}
}
OutlinedButton(onClick = { monero = true }, Modifier.fillMaxWidth()) {
Text("Monero")
}
TextButton(onClick = { DonatePrompt.answer(context, donated = false) }, Modifier.fillMaxWidth()) {
Text("Not now")
}
}
},
)
}
@@ -16,11 +16,9 @@ import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.statusBars
import androidx.compose.foundation.layout.windowInsetsPadding
import androidx.compose.material3.AlertDialog
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
@@ -39,7 +37,7 @@ import app.seb3thehacker.gearslip.BuildConfig
import app.seb3thehacker.gearslip.CertProvider
import app.seb3thehacker.gearslip.stats.UsageStats
private enum class Screen { HOME, LOGS, SETTINGS, CAR_PREVIEW, CERT_SETUP, SETUP_GUIDE, USAGE_NOTES, HELP, WHATS_NEW, RECOMMENDED_APPS }
private enum class Screen { INTRO, HOME, LOGS, SETTINGS, CAR_PREVIEW, CERT_SETUP, SETUP_GUIDE, USAGE_NOTES, HELP, WHATS_NEW, RECOMMENDED_APPS }
/** Three screens and a back stack of depth one: no navigation library needed. */
@Composable
@@ -49,19 +47,29 @@ fun GearslipApp(onDisconnect: () -> Unit, onRequestCallScreening: () -> Unit) {
!AppSettings.hasSkippedCertSetup(context) && !CertProvider.hasAnyCert(context)
}
val needsSetupGuide = remember { !AppSettings.hasSeenPermissionsSetup(context) }
var screen by rememberSaveable {
mutableStateOf(
when {
needsCertSetup -> Screen.CERT_SETUP
needsSetupGuide -> Screen.SETUP_GUIDE
// Set up before usage notes existed: ask once, on the setup guide's own page.
!UsageStats.asked(context) -> Screen.USAGE_NOTES
else -> Screen.HOME
},
)
// Where launch goes once the intro (if any) is done.
val afterIntro = remember {
when {
needsCertSetup -> Screen.CERT_SETUP
needsSetupGuide -> Screen.SETUP_GUIDE
// Set up before usage notes existed: ask once, on the setup guide's own page.
!UsageStats.asked(context) -> Screen.USAGE_NOTES
else -> Screen.HOME
}
}
BackHandler(enabled = screen != Screen.HOME) {
screen = if (screen == Screen.RECOMMENDED_APPS) Screen.SETTINGS else Screen.HOME
var screen by rememberSaveable { mutableStateOf(if (Intro.due(context)) Screen.INTRO else afterIntro) }
// A replay from About goes back to Settings; the launch one carries on into setup.
var introReturn by rememberSaveable { mutableStateOf(afterIntro) }
// Home's "it worked" card opens Settings at usage notes rather than at the top.
var settingsAtUsageNotes by rememberSaveable { mutableStateOf(false) }
// What's new and Connection help open from Home and from Settings; back returns to whichever.
var backTo by rememberSaveable { mutableStateOf(Screen.SETTINGS) }
BackHandler(enabled = screen != Screen.HOME && screen != Screen.INTRO) {
screen = when (screen) {
Screen.RECOMMENDED_APPS, Screen.LOGS, Screen.CAR_PREVIEW -> Screen.SETTINGS
Screen.HELP, Screen.WHATS_NEW -> backTo
else -> Screen.HOME
}
}
// Home stays upright; every other screen turns with the phone.
@@ -75,6 +83,10 @@ fun GearslipApp(onDisconnect: () -> Unit, onRequestCallScreening: () -> Unit) {
// The guide asks for this now, but anyone who finished it before that never saw the page.
val notificationPermission = rememberLauncherForActivityResult(ActivityResultContracts.RequestPermission()) {}
// Reads the certificate once in the background, so Settings has it ready when opened.
LaunchedEffect(Unit) {
kotlinx.coroutines.withContext(kotlinx.coroutines.Dispatchers.Default) { runCatching { CertSummary.read(context) } }
}
LaunchedEffect(Unit) {
if (!needsSetupGuide &&
context.checkSelfPermission(Manifest.permission.POST_NOTIFICATIONS) != PackageManager.PERMISSION_GRANTED
@@ -89,29 +101,35 @@ fun GearslipApp(onDisconnect: () -> Unit, onRequestCallScreening: () -> Unit) {
.windowInsetsPadding(WindowInsets.statusBars)
.consumeWindowInsets(WindowInsets.statusBars),
) {
if (BuildConfig.DEBUG && screen != Screen.CAR_PREVIEW) DevBuildBanner()
if (BuildConfig.DEBUG && screen != Screen.CAR_PREVIEW && screen != Screen.INTRO) DebugBuildBanner()
Column(Modifier.weight(1f)) {
when (screen) {
Screen.INTRO -> IntroScreen(onDone = { screen = introReturn })
Screen.HOME -> HomeScreen(
onOpenLogs = { screen = Screen.LOGS },
onOpenSettings = { screen = Screen.SETTINGS },
onOpenCarPreview = { screen = Screen.CAR_PREVIEW },
onOpenHelp = { screen = Screen.HELP },
onOpenSettings = { settingsAtUsageNotes = false; screen = Screen.SETTINGS },
onOpenWhatsNew = { backTo = Screen.HOME; screen = Screen.WHATS_NEW },
onOpenHelp = { backTo = Screen.HOME; screen = Screen.HELP },
onOpenUsageNotes = { settingsAtUsageNotes = true; screen = Screen.SETTINGS },
onDisconnect = onDisconnect,
)
Screen.WHATS_NEW -> WhatsNewScreen(onBack = { screen = Screen.SETTINGS })
Screen.LOGS -> LogsScreen(onBack = { screen = Screen.HOME })
Screen.WHATS_NEW -> WhatsNewScreen(onBack = { screen = backTo })
Screen.LOGS -> LogsScreen(onBack = { screen = Screen.SETTINGS })
Screen.SETTINGS -> SettingsScreen(
onBack = { screen = Screen.HOME },
onOpenRecommendedApps = { screen = Screen.RECOMMENDED_APPS },
onOpenWhatsNew = { screen = Screen.WHATS_NEW },
onOpenRecommendedApps = { settingsAtUsageNotes = false; screen = Screen.RECOMMENDED_APPS },
onOpenWhatsNew = { settingsAtUsageNotes = false; backTo = Screen.SETTINGS; screen = Screen.WHATS_NEW },
onOpenHelp = { settingsAtUsageNotes = false; backTo = Screen.SETTINGS; screen = Screen.HELP },
onOpenCarPreview = { settingsAtUsageNotes = false; screen = Screen.CAR_PREVIEW },
onOpenLogs = { settingsAtUsageNotes = false; screen = Screen.LOGS },
onReplayTutorial = { screen = Screen.SETUP_GUIDE },
onReplayIntro = { Intro.replay(context); introReturn = Screen.SETTINGS; screen = Screen.INTRO },
atUsageNotes = settingsAtUsageNotes,
)
Screen.RECOMMENDED_APPS -> RecommendedAppsScreen(onBack = { screen = Screen.SETTINGS })
Screen.CAR_PREVIEW -> CarPreviewScreen(onBack = { screen = Screen.HOME })
Screen.CAR_PREVIEW -> CarPreviewScreen(onBack = { screen = Screen.SETTINGS })
Screen.HELP -> HelpScreen(
onBack = { screen = Screen.HOME },
onBack = { screen = backTo },
onRequestCallScreening = onRequestCallScreening,
)
Screen.CERT_SETUP -> CertSetupScreen(
@@ -133,8 +151,6 @@ fun GearslipApp(onDisconnect: () -> Unit, onRequestCallScreening: () -> Unit) {
// A car keyboard is open: offer the phone's own, for a passenger.
if (screen != Screen.CAR_PREVIEW) PhoneTypingBar()
}
CompatWarning()
}
/**
@@ -144,7 +160,7 @@ fun GearslipApp(onDisconnect: () -> Unit, onRequestCallScreening: () -> Unit) {
* never mistaken for a release one.
*/
@Composable
private fun DevBuildBanner() {
private fun DebugBuildBanner() {
Surface(
modifier = Modifier.fillMaxWidth().height(28.dp),
color = Color(0xFFF5C518),
@@ -152,36 +168,10 @@ private fun DevBuildBanner() {
) {
Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
Text(
"DEV BUILD",
"DEBUG BUILD",
style = MaterialTheme.typography.labelSmall,
fontWeight = FontWeight.Bold,
)
}
}
}
/** Shown once, on first run: the certificate is known to fail on head units newer than ~2020. */
@Composable
private fun CompatWarning() {
val context = LocalContext.current
var visible by rememberSaveable { mutableStateOf(!AppSettings.hasSeenCompatWarning(context)) }
if (visible) {
AlertDialog(
onDismissRequest = {},
title = { Text("Before you plug in") },
text = {
Text(
"Gearslip works with cars built before 2020. " +
"Newer firmware often blocks the connection.",
)
},
confirmButton = {
TextButton(onClick = {
AppSettings.setSeenCompatWarning(context)
visible = false
}) { Text("Got it") }
},
)
}
}
@@ -1,6 +1,7 @@
package app.seb3thehacker.gearslip.ui
import app.seb3thehacker.gearslip.BuildConfig
import app.seb3thehacker.gearslip.car.CarSettings
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
@@ -23,13 +24,16 @@ import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.TopAppBar
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
/**
* One step of the setup list: a sentence that says what to do, and optionally how.
* One step: a sentence that says what to do, and optionally how.
*
* [command] is for a step that cannot be done on the phone; it is shown as something to copy
* rather than to read, because a mistyped package name fails silently. [onClick] is for a step
@@ -43,51 +47,58 @@ private class Step(
val onClick: (() -> Unit)? = null,
)
private fun setupSteps(onRequestCallScreening: () -> Unit) = listOf(
Step("Use a car from 2020 or earlier.", "Newer head units reject the certificate."),
private val FIRST_DRIVE = listOf(
Step("Turn off Android Auto.", "Disable or uninstall the Android Auto app, so the car talks to Gearslip instead."),
Step("Load a certificate.", "In Settings, download one or import your own."),
Step(
"On GrapheneOS, exempt Gearslip from exploit protections.",
"Open Settings, then Apps, then Gearslip, and turn on Exploit protection compatibility mode.",
"Allow the microphone.",
"Android won't let an app pass music to the car without it. Gearslip listens only when " +
"you tap the mic or press the talk button on your wheel.",
),
Step("Disable or uninstall the Android Auto app."),
Step("Load a certificate.", "Open Settings, then download or import a certificate."),
)
private fun inTheCar(onRequestCallScreening: () -> Unit) = listOf(
Step(
"Allow the microphone, so music can reach the car.",
"Android asks for the microphone before it will let any app pass music along. " +
"Gearslip listens only when you tap Reply or press the voice button. The rest of the " +
"time it copies what a music app is playing, and nothing else.",
"Plug in, or connect wirelessly.",
"Gearslip opens by itself when the car connects. For wireless, pair the phone with the " +
"car first, then tap Connect wirelessly on Home.",
),
Step(
"Optional: stop Android asking about audio every drive.",
"Connect the phone to a computer with USB debugging turned on, and run this command. " +
"It tells Android to trust Gearslip with audio from now on. Skip it and everything " +
"still works, but you have to tap Start once each time you set off.",
command = "adb shell appops set ${BuildConfig.APPLICATION_ID} PROJECT_MEDIA allow",
"Pair over Bluetooth for calls.",
"Calls play through the car once your phone and the car are paired. Gearslip offers to " +
"pair the first time you plug in.",
),
Step(
"Optional: keep notifications readable while music plays.",
"Android treats sending audio to the car like sharing your screen, and hides what your " +
"notifications say. Run this command too, and Gearslip switches that off only while " +
"music is going to the car, then puts it back.",
command = "adb shell pm grant ${BuildConfig.APPLICATION_ID} android.permission.WRITE_SECURE_SETTINGS",
),
Step(
"Optional: let the car screen answer and decline calls.",
"This sets Gearslip as your phone's caller ID and spam-blocking app, the same kind of " +
"consent a call-blocking app asks for. Without it, calls still ring, but only the " +
"phone itself can answer or decline them.",
"Answer calls from the car screen.",
"Set Gearslip as your caller ID and spam app. Without it, calls still ring, but only " +
"the phone can answer them.",
buttonLabel = "Set up",
onClick = onRequestCallScreening,
),
Step("Plug the phone into the car.", "Gearslip opens by itself when the head unit connects."),
)
/** The full setup reference, behind the help button: what the one-time guide walks through, kept
* here too so it can be reread without repeating the guide itself. */
private val USB_AUDIO = listOf(
Step(
"Stop the audio prompt each drive.",
"Connect the phone to a computer with USB debugging on, and run this once. Skip it and " +
"you tap Start each time you set off.",
command = "adb shell appops set ${BuildConfig.APPLICATION_ID} PROJECT_MEDIA allow",
),
Step(
"Keep notifications readable.",
"Android hides what notifications say while it shares audio. Run this too, and Gearslip " +
"lifts that only while music goes to the car.",
command = "adb shell pm grant ${BuildConfig.APPLICATION_ID} android.permission.WRITE_SECURE_SETTINGS",
),
)
/** Which cars work, and the setup the one-time guide walks through, to reread at any time. */
@OptIn(ExperimentalMaterial3Api::class)
@Composable
fun HelpScreen(onBack: () -> Unit, onRequestCallScreening: () -> Unit) {
val steps = remember(onRequestCallScreening) { setupSteps(onRequestCallScreening) }
val car = remember(onRequestCallScreening) { inTheCar(onRequestCallScreening) }
val usb by CarSettings.pipeAudio.collectAsState()
val context = LocalContext.current
Scaffold(
containerColor = MaterialTheme.colorScheme.background,
@@ -108,39 +119,59 @@ fun HelpScreen(onBack: () -> Unit, onRequestCallScreening: () -> Unit) {
.padding(padding)
.padding(horizontal = 24.dp)
.verticalScroll(rememberScrollState()),
verticalArrangement = Arrangement.spacedBy(24.dp),
) {
Spacer(Modifier.height(8.dp))
Spacer(Modifier.height(0.dp))
SettingsSection("Which cars work") {
Text(
"Most cars work. Starting around 2020, some cars added newer security " +
"standards that reject Gearslip's certificate, and Gearslip can't run in those cars.",
style = MaterialTheme.typography.bodyLarge,
modifier = Modifier.padding(start = 16.dp, end = 16.dp, top = 16.dp),
)
FilledTonalButton(
onClick = { GearslipLinks.open(context, GearslipLinks.CARS) },
modifier = Modifier.padding(16.dp),
) { Text("See the full list") }
}
Steps("Before your first drive", FIRST_DRIVE)
Steps("In the car", car)
if (usb) Steps("USB audio", USB_AUDIO)
Spacer(Modifier.height(16.dp))
Column(verticalArrangement = Arrangement.spacedBy(18.dp)) {
steps.forEachIndexed { index, step ->
Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
Text(
"${index + 1}",
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.Bold,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.width(16.dp),
)
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
Text(step.action, style = MaterialTheme.typography.bodyLarge, fontWeight = FontWeight.SemiBold)
step.detail?.let {
Text(
it,
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
step.command?.let { CopyableCommand(it) }
if (step.onClick != null) {
FilledTonalButton(onClick = step.onClick, modifier = Modifier.padding(top = 4.dp)) {
Text(step.buttonLabel ?: "Open")
}
}
}
}
}
/** A titled card of numbered steps. */
@Composable
private fun Steps(title: String, steps: List<Step>) {
SettingsSection(title) {
steps.forEachIndexed { index, step ->
if (index > 0) SettingsDivider()
Row(
Modifier.fillMaxWidth().padding(16.dp),
horizontalArrangement = Arrangement.spacedBy(16.dp),
) {
Text(
"${index + 1}",
style = MaterialTheme.typography.titleMedium,
fontWeight = FontWeight.Bold,
color = MaterialTheme.colorScheme.primary,
modifier = Modifier.width(16.dp),
)
Column(verticalArrangement = Arrangement.spacedBy(4.dp)) {
Text(step.action, style = MaterialTheme.typography.bodyLarge, fontWeight = FontWeight.SemiBold)
step.detail?.let {
Text(it, style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
step.command?.let { CopyableCommand(it) }
if (step.onClick != null) {
FilledTonalButton(onClick = step.onClick, modifier = Modifier.padding(top = 4.dp)) {
Text(step.buttonLabel ?: "Open")
}
}
}
}
Spacer(Modifier.height(32.dp))
}
}
}
@@ -1,7 +1,12 @@
package app.seb3thehacker.gearslip.ui
import app.seb3thehacker.gearslip.speech.SpeechModels
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
@@ -11,6 +16,13 @@ import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size
import android.content.Context
import androidx.compose.material3.ButtonDefaults
import androidx.compose.ui.graphics.Color
import androidx.compose.foundation.isSystemInDarkTheme
import app.seb3thehacker.gearslip.car.CarSettings
import app.seb3thehacker.gearslip.car.BluetoothAudioCheck
import android.provider.Settings
import android.content.Intent
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
@@ -20,6 +32,7 @@ import androidx.compose.material.icons.filled.Send
import androidx.compose.material.icons.filled.List
import androidx.compose.material.icons.filled.PlayArrow
import androidx.compose.material.icons.filled.Settings
import androidx.compose.material.icons.filled.Star
import androidx.compose.material3.Button
import androidx.compose.material3.CircularProgressIndicator
import androidx.compose.material3.HorizontalDivider
@@ -31,6 +44,7 @@ import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.vector.ImageVector
@@ -41,117 +55,163 @@ import androidx.lifecycle.compose.collectAsStateWithLifecycle
import app.seb3thehacker.gearslip.BuildConfig
import app.seb3thehacker.gearslip.SessionStatus
import app.seb3thehacker.gearslip.notify.TestMessage
import app.seb3thehacker.gearslip.stats.WorkedPrompt
@Composable
fun HomeScreen(
onOpenLogs: () -> Unit,
onOpenSettings: () -> Unit,
onOpenCarPreview: () -> Unit,
onOpenWhatsNew: () -> Unit,
onOpenHelp: () -> Unit,
onOpenUsageNotes: () -> Unit,
onDisconnect: () -> Unit,
) {
val status by SessionStatus.state.collectAsStateWithLifecycle()
val context = LocalContext.current
val workedCar by remember { WorkedPrompt.pending(context) }.collectAsStateWithLifecycle()
val live = status.phase == SessionStatus.Phase.CONNECTING ||
status.phase == SessionStatus.Phase.PROJECTING
Scaffold(containerColor = MaterialTheme.colorScheme.background) { padding ->
Column(
Modifier
.fillMaxSize()
.padding(padding)
.padding(horizontal = 24.dp),
) {
val celebrate by WorkedPrompt.celebrate.collectAsStateWithLifecycle()
Box {
Scaffold(containerColor = MaterialTheme.colorScheme.background) { padding ->
Column(
Modifier
.weight(1f)
.verticalScroll(rememberScrollState()),
.fillMaxSize()
.padding(padding)
.padding(horizontal = 24.dp),
) {
Spacer(Modifier.height(24.dp))
Row(
Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
Column(
Modifier
.weight(1f)
.verticalScroll(rememberScrollState()),
) {
Text(
"Gearslip",
style = MaterialTheme.typography.displayLarge,
fontWeight = FontWeight.Bold,
color = MaterialTheme.colorScheme.onBackground,
)
IconButton(
onClick = onOpenSettings,
modifier = Modifier.size(56.dp),
Spacer(Modifier.height(24.dp))
Row(
Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.SpaceBetween,
) {
Icon(Icons.Filled.Settings, contentDescription = "Settings", modifier = Modifier.size(40.dp))
Text(
"Gearslip",
style = MaterialTheme.typography.displayLarge,
fontWeight = FontWeight.Bold,
color = MaterialTheme.colorScheme.onBackground,
)
IconButton(
onClick = onOpenSettings,
modifier = Modifier.size(56.dp),
) {
Icon(Icons.Filled.Settings, contentDescription = "Settings", modifier = Modifier.size(40.dp))
}
}
}
Text(
"v${LocalContext.current.appVersionName()}",
style = MaterialTheme.typography.labelLarge,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(16.dp))
StatusCard(status)
Spacer(Modifier.height(16.dp))
}
// Sits at the bottom, where a thumb reaches it.
Column(Modifier.padding(vertical = 16.dp)) {
val live = status.phase == SessionStatus.Phase.CONNECTING ||
status.phase == SessionStatus.Phase.PROJECTING
if (live) {
Button(onClick = onDisconnect, modifier = Modifier.fillMaxWidth().height(56.dp)) {
Text("Disconnect")
Text(
"v${LocalContext.current.appVersionName()}",
style = MaterialTheme.typography.labelLarge,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
Spacer(Modifier.height(16.dp))
StatusCard(status)
if (live) BluetoothMissingCard()
VoiceDownloadCard()
// The first drive that worked asks once to share it, right under the result.
val car = workedCar
if (car != null && !live) {
Spacer(Modifier.height(16.dp))
WorkedCard(car, onSeeWhatsSent = onOpenUsageNotes)
}
Spacer(Modifier.height(16.dp))
UpdateCard()
}
Surface(color = MaterialTheme.colorScheme.surfaceContainerHigh, shape = MaterialTheme.shapes.extraLarge) {
Column {
if (!live) {
ActionRow(Icons.Filled.PlayArrow, "Car preview", onOpenCarPreview)
RowDivider()
}
ActionRow(Icons.Filled.List, "Live logs", onOpenLogs)
RowDivider()
ActionRow(Icons.Filled.Info, "Connection help", onOpenHelp)
if (BuildConfig.DEBUG) {
val context = LocalContext.current
RowDivider()
ActionRow(Icons.Filled.Send, "Send a test message") { TestMessage.post(context) }
// Sits at the bottom, where a thumb reaches it.
Column(Modifier.padding(vertical = 16.dp)) {
SettingsCard {
SettingsRow(
"What's new in ${Intro.release}",
Modifier.clickable(onClick = onOpenWhatsNew),
icon = Icons.Filled.Star,
trailing = { Arrow() },
)
SettingsDivider()
SettingsRow(
"Which cars work",
Modifier.clickable(onClick = onOpenHelp),
subtitle = "Most do. See which ones, and what to set up first",
icon = Icons.Filled.Info,
trailing = { Arrow() },
)
}
if (live) {
Spacer(Modifier.height(16.dp))
Button(onClick = onDisconnect, modifier = Modifier.fillMaxWidth().height(56.dp)) {
Text("Disconnect")
}
}
}
}
}
// Sharing just went on from the "it worked" ask: a little thanks.
if (celebrate) ConfettiFall(onDone = WorkedPrompt::celebrated)
// The usage notes ask comes first; the donation ask waits until that's answered.
if (workedCar == null) DonateDialog(live)
}
}
/** Bluetooth audio is chosen, but the phone isn't connected to the car. See [BluetoothAudioCheck]. */
@Composable
private fun BluetoothMissingCard() {
val missing by BluetoothAudioCheck.missing.collectAsStateWithLifecycle()
val usb by CarSettings.pipeAudio.collectAsStateWithLifecycle()
val warn by CarSettings.bluetoothWarning.collectAsStateWithLifecycle()
if (!missing || usb || !warn) return
val context = LocalContext.current
// Yellow, not the theme's error red: music is missing, but nothing is broken. Material has no
// warning colour, so these are fixed amber tones for light and dark.
val dark = isSystemInDarkTheme()
val container = if (dark) Color(0xFF4A3B00) else Color(0xFFFFE08A)
val content = if (dark) Color(0xFFFFE08A) else Color(0xFF3A2E00)
Spacer(Modifier.height(16.dp))
Surface(color = container, contentColor = content, shape = MaterialTheme.shapes.extraLarge) {
Column(Modifier.fillMaxWidth().padding(20.dp), verticalArrangement = Arrangement.spacedBy(8.dp)) {
Text("Bluetooth isn't connected", style = MaterialTheme.typography.titleLarge, fontWeight = FontWeight.SemiBold)
Text(
"Music plays over Bluetooth, but your phone isn't connected to the car. Connect it " +
"here or from the car's Bluetooth menu.",
style = MaterialTheme.typography.bodyMedium,
)
Button(
colors = ButtonDefaults.buttonColors(containerColor = content, contentColor = container),
onClick = {
runCatching {
context.startActivity(Intent(Settings.ACTION_BLUETOOTH_SETTINGS).addFlags(Intent.FLAG_ACTIVITY_NEW_TASK))
}
},
modifier = Modifier.padding(top = 4.dp),
) { Text("Bluetooth settings") }
}
}
}
/** While the voice models download, how far along they are; nothing otherwise. */
@Composable
private fun VoiceDownloadCard() {
val progress by SpeechModels.progress.collectAsStateWithLifecycle()
val percent = progress ?: return
Spacer(Modifier.height(16.dp))
SettingsCard {
Column(Modifier.padding(horizontal = 16.dp, vertical = 14.dp), verticalArrangement = Arrangement.spacedBy(10.dp)) {
Text("Downloading voice, $percent%", style = MaterialTheme.typography.bodyLarge)
LinearProgressIndicator(progress = { percent / 100f }, modifier = Modifier.fillMaxWidth())
}
}
}
@Composable
private fun ActionRow(icon: ImageVector, label: String, onClick: () -> Unit) {
Row(
Modifier
.fillMaxWidth()
.clickable(onClick = onClick)
.padding(horizontal = 20.dp, vertical = 18.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(16.dp),
) {
Icon(icon, contentDescription = null, tint = MaterialTheme.colorScheme.primary)
Text(label, style = MaterialTheme.typography.titleMedium, modifier = Modifier.weight(1f))
Icon(
Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
}
}
@Composable
private fun RowDivider() {
HorizontalDivider(
modifier = Modifier.padding(horizontal = 20.dp),
color = MaterialTheme.colorScheme.outlineVariant,
)
}
private fun Arrow() = Icon(
Icons.AutoMirrored.Filled.KeyboardArrowRight,
contentDescription = null,
tint = MaterialTheme.colorScheme.onSurfaceVariant,
)
@Composable
private fun StatusCard(status: SessionStatus.Snapshot) {
@@ -164,11 +224,11 @@ private fun StatusCard(status: SessionStatus.Snapshot) {
scheme.surfaceContainerHigh to scheme.onSurface
}
val live = status.phase == SessionStatus.Phase.CONNECTING || status.phase == SessionStatus.Phase.PROJECTING
Surface(color = container, contentColor = content, shape = MaterialTheme.shapes.extraLarge) {
Column(Modifier.fillMaxWidth().padding(24.dp)) {
Row(
Modifier
.fillMaxWidth()
.padding(20.dp),
Modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(16.dp),
) {
@@ -184,11 +244,17 @@ private fun StatusCard(status: SessionStatus.Snapshot) {
) {}
}
Column {
Text(status.headline, style = MaterialTheme.typography.titleLarge, fontWeight = FontWeight.SemiBold)
Text(status.headline, style = MaterialTheme.typography.headlineSmall, fontWeight = FontWeight.SemiBold)
Spacer(Modifier.height(4.dp))
Text(status.detail, style = MaterialTheme.typography.bodyMedium)
}
}
// The other way in: wireless, right under the cable's instructions.
if (!live) {
Spacer(Modifier.height(20.dp))
WirelessButton()
}
}
}
}
Loaded 100 of 126 files, more files were not shown because too many files have changed in this diff. Show more