Author SHA1 Message Date
Seb3thehacker e8f51f4947 0.3.01-alpha: wireless around VPNs and early Wi-Fi details, re-signed apps lose the red X 2026-10-15 12:38:06 -07:00
Seb3thehacker 2281347728 links: the repo now lives at twinksoftware/gearslip 2026-10-15 12:38:06 -07:00
Seb3thehacker 6d08fbd871 license: Gearslip now belongs to Twink Software LLC 2026-10-15 12:38:06 -07:00
Seb3thehacker 1322cbaf39 readme: package name and certificate hash for AppVerifier 2026-10-15 08:04:28 -07:00
Seb3thehacker db6f1b8e82 launcher: a patched Waze, signed with someone else's key, loses the red X and can start as the map app 2026-10-15 07:50:34 -07:00
Seb3thehacker 0177975bc8 wireless: wait for the address when the Wi-Fi details come first, and work around or name a VPN that blocks the car 2026-10-15 07:25:59 -07:00
Seb3thehacker 5f30e7b014 readme: an eighth screenshot, coffee nearby in Vela 2026-10-15 07:21:47 -07:00
Seb3thehacker c42406f353 support: Liberapay first again, then Buy Me a Coffee, Monero last 2026-10-14 23:41:34 -07:00
Seb3thehacker e8d025df47 support: say Buy Me a Coffee keeps 5% 2026-10-14 23:39:05 -07:00
Seb3thehacker c6d8d8e787 support: Buy Me a Coffee first in Settings, the popup, the thank-you page and the repo's Sponsor button; Monero last 2026-10-14 23:36:30 -07:00
Seb3thehacker 77b84212db readme: new screenshots for 0.3, with the assistant, a route in the bar, and the song title 2026-10-14 23:34:35 -07:00
Seb3thehacker b12e9651c1 wireless: when Android won't ask for the permissions, a button to enable Nearby devices in the app's settings 2026-10-14 22:06:24 -07:00
Seb3thehacker a5c46967b7 voice: drop the unused archive download and commons-compress with it, which CodeQL flagged 2026-10-14 21:11:40 -07:00
Seb3thehacker 30a4b65a08 release: notes list calls as a known issue, and new releases start as pre-releases 2026-10-14 19:57:32 -07:00
Seb3thehacker 88ff5b0606 voice: Parakeet small downloads as plain files, like the other models, instead of an archive the phone unpacks slowly 2026-10-14 19:56:34 -07:00
Seb3thehacker aa15884897 Merge branch 'experimental' 2026-10-14 19:49:37 -07:00
Seb3thehacker 84e1145459 intro: drop the Let's drive page, so it ends on thanks 2026-10-14 19:45:47 -07:00
Seb3thehacker 53de4a2512 0.3.00-alpha, with its notes in the app and on the GitHub release 2026-10-14 19:40:25 -07:00
Seb3thehacker ed46868d9b voice: say Canary knows English, Spanish, German and French, and that Gearslip listens in English for now 2026-10-14 19:36:23 -07:00
Seb3thehacker 2bf4753125 support: Monero, in a popup with the full address, Open in wallet and Copy address 2026-10-14 19:29:57 -07:00
Seb3thehacker 879b428d1f support: a Liberapay button on the intro's thank-you page, no "through a nonprofit", and room for Monero and a give-once card link, hidden until they're set 2026-10-14 19:21:41 -07:00
Seb3thehacker 35bd239c2f support: Liberapay only until GitHub Sponsors is approved, and "support" instead of "donate", since giving to the LLC isn't tax-deductible 2026-10-14 19:17:19 -07:00
Seb3thehacker 815a75ad6d car: say when the phone's location is off, ten seconds into a drive, with Don't show again and a setting to turn the warning off 2026-10-14 15:45:10 -07:00
Seb3thehacker 9b7734bec8 voice: "navigate to X" picks the nearest X in the map app's own search and stops at its start screen; it no longer sends the phone's position, which Organic Maps took as the destination 2026-10-14 15:45:10 -07:00
Seb3thehacker 7ffa203221 voice: a new destination mid-route ends the old route first instead of being ignored; switching or closing map apps ends the old app's route 2026-10-14 14:24:47 -07:00
Seb3thehacker 9f982875fe release builds shrink with R8: 65 MB down to 30 MB, small enough for IzzyOnDroid. -Pr8 does the same to a debug build for testing 2026-10-14 13:29:00 -07:00
Seb3thehacker 5c799fed38 Change Liberapay entry to username only
Updated Liberapay username to remove URL.
2026-10-14 12:53:27 -07:00
Seb3thehacker 249e6cefed Add Liberapay username to FUNDING.yml
Updated Liberapay entry with a specific username.
2026-10-14 12:51:45 -07:00
Seb3thehacker 3c9603175e debug builds send usage notes too, marked debug so the server keeps them out of the stats; call them debug builds everywhere instead of dev 2026-10-14 12:25:45 -07:00
Seb3thehacker 10dfb46a96 player: draw the app's own like, shuffle and repeat buttons, lit when their icon says they're on (Tidal showed every song liked and shuffle off) 2026-10-14 00:42:04 -07:00
Seb3thehacker e587ea97e1 intro: tell updates from new installs by setup having been done, so clearing data counts as new 2026-10-13 19:11:33 -07:00
Seb3thehacker daa6f24289 intro: new page order ending on thanks, a plainer welcome, and it resumes where it was after the app restarts 2026-10-13 19:08:48 -07:00
Seb3thehacker 76955669f5 intro: only after an update, never on a new install; recommend Organic Maps before Vela and ViVi Music before Metrolist; the thanks page links the list of cars that work 2026-10-13 19:00:51 -07:00
Seb3thehacker cd7b13c88e intro: Back and Next buttons like the setup guide, instead of tapping anywhere 2026-10-13 18:15:26 -07:00
Seb3thehacker 3d642c17c7 home: the Bluetooth warning is yellow, not red 2026-10-13 18:12:42 -07:00
Seb3thehacker cce3b50b52 music: an app whose library never answers falls back to its playback controls after 8 seconds, instead of spinning for good 2026-10-13 18:09:04 -07:00
Seb3thehacker 88e1885b2d car audio: with Bluetooth audio chosen, warn on the car and on Home when the phone isn't connected to the car 20 seconds in 2026-10-13 18:09:04 -07:00
Seb3thehacker a84d36f17a voice: app names match however the recognizer spaces them, so "with map quest" picks MapQuest 2026-10-13 17:57:30 -07:00
Seb3thehacker 8b269f4734 voice: opening a screen mid-download no longer deletes the download in progress 2026-10-13 17:54:36 -07:00
Seb3thehacker 3042e01e75 donations: at the top of Settings, and a popup on Home after the third drive that worked, then at most every three months 2026-10-13 17:42:05 -07:00
Seb3thehacker 71397a0777 settings: donation links are back, in their own section above About 2026-10-13 17:39:15 -07:00
Seb3thehacker cae05ecdc0 car launcher: only apps that don't work get a mark, a red X, and the badge key is gone 2026-10-13 17:39:15 -07:00
Seb3thehacker 73be554eb2 car audio: pause the phone's music when the car switches to the radio, send AAC to cars that take nothing else, and open GM's own channel in GM cars 2026-10-13 17:14:32 -07:00
Seb3thehacker 2812613bca wireless: say the car must be paired first, with a button to Bluetooth settings 2026-10-13 17:14:32 -07:00
Seb3thehacker 014942338b usage notes: shorter intro and options, so all four fit on screen 2026-10-13 17:04:47 -07:00
Seb3thehacker 3c81980385 usage notes: name every field each option sends 2026-10-13 16:58:46 -07:00
Seb3thehacker c029a0894a voice: one button downloads Kroko and Canary in setup, with progress on Home and in a silent notification 2026-10-13 16:56:56 -07:00
Seb3thehacker eba8ac1127 home: What's new and which cars work sit at the bottom 2026-10-13 16:45:59 -07:00
Seb3thehacker 1086e7c8fa cars: a list of cars drivers have confirmed, linked from the README and the help screen 2026-10-13 16:40:21 -07:00
Seb3thehacker 470ec2f093 usage notes: say whether each drive was USB or wireless, and whether music went over USB or Bluetooth 2026-10-13 16:24:28 -07:00
Seb3thehacker 341e62d1d8 readme: notes say which boxes are ticked 2026-10-13 16:17:49 -07:00
Seb3thehacker 2891a170db usage notes: say which boxes are ticked, so the server can drop details from unticked ones 2026-10-13 16:17:49 -07:00
Seb3thehacker 3855cc4781 readme: wireless, voice, the Bluetooth or USB audio choice, and updates 2026-10-13 16:13:02 -07:00
Seb3thehacker f1a6901fbc usage notes: ask after the first good drive only if setup never asked 2026-10-13 16:11:43 -07:00
Seb3thehacker b848c5d910 home: What's new and which cars work under the status card; the usage notes switch stays in Settings 2026-10-13 16:06:26 -07:00
Seb3thehacker 4f1e6dcbfa wireless: Disconnect works while the car hasn't answered yet 2026-10-13 16:00:40 -07:00
Seb3thehacker 10e8568aba intro: thank note senders for confirming cars newer than 2020 2026-10-13 15:55:29 -07:00
Seb3thehacker 724dcf25c7 home: put Connect wirelessly in the status card, with the usage notes switch always under it 2026-10-13 15:48:54 -07:00
Seb3thehacker dd1fd0242f home: put Connect wirelessly in the status card, as a full button 2026-10-13 15:41:59 -07:00
Seb3thehacker 6faad9fdd5 help: say most cars work, group the steps, and show the adb steps only for USB audio 2026-10-13 15:39:20 -07:00
Seb3thehacker 3869258e0e home: show only the car's status; help, preview, logs and the dev test message move to Settings 2026-10-13 15:38:40 -07:00
Seb3thehacker 7d9eb20ff0 updates: hide the channel picker until a pre-release ships 2026-10-13 15:27:15 -07:00
Seb3thehacker 8fd840bd8d updates: offer the final release to people on its beta or alpha 2026-10-13 15:21:17 -07:00
Seb3thehacker 5947bc1e63 intro: add a step inviting drivers to the Matrix chat 2026-10-13 15:21:17 -07:00
Seb3thehacker e13b0618db voice: recommend Canary for the final words, at a third of Parakeet large's size 2026-10-13 15:15:41 -07:00
Seb3thehacker 9100dac290 settings: tidy the order, fold updates into one row with Stable, Beta and Alpha, and drop the donation asks and the after-drive notification 2026-10-13 15:05:45 -07:00
Seb3thehacker f1fe94c487 bluetooth: offer to pair with the car on first connect, and pick Bluetooth or USB audio in setup 2026-10-13 14:55:11 -07:00
Seb3thehacker 5fe59d350c wireless: send about a third of the cable bitrate at 30fps, so the video keeps up and the music stops cutting out 2026-10-13 14:31:03 -07:00
Seb3thehacker 909c3549d6 wireless: ask for Android 17's local network permission, without which the car never hears from the phone 2026-10-13 14:12:13 -07:00
Seb3thehacker 1514721498 wireless: connect over the phone's main Wi-Fi when it's already on the car's network, and log why traffic is held back 2026-10-13 14:03:55 -07:00
Seb3thehacker 4000e0a9e9 wireless: keep trying over the phone's own Wi-Fi when Android drops the car's network 2026-10-13 13:19:22 -07:00
Seb3thehacker 02d6cb546e Merge branch 'experimental' into wireless 2026-10-13 12:41:09 -07:00
Seb3thehacker 59f323ec06 directions: speak each turn on the car's guidance channel
Map apps' own voice prompts can't reach the car over USB, so Gearslip reads
the turn the cluster already gets and says it through the car's directions
sink, which ducks the music. On by default, with a switch in car settings.
2026-10-13 12:26:31 -07:00
Seb3thehacker 0d0bfd3d66 setup: the 0.3 look, with a badge, a big headline and shorter words on every step 2026-10-13 01:28:54 -07:00
Seb3thehacker b3f2088a1d intro: pick the car's button style, with sample buttons that redraw as you choose 2026-10-13 01:14:06 -07:00
Seb3thehacker 01a41c01c8 voice: "pause" keeps the music paused after the mic lets go 2026-10-13 00:52:00 -07:00
Seb3thehacker 62b1835444 theme: flat is the default 2026-10-13 00:45:39 -07:00
Seb3thehacker 1cd3d6f856 car mic: open a voice session and name the codec before asking for the car's microphone 2026-10-13 00:37:49 -07:00
Seb3thehacker 8912e88406 voice: "navigate to the nearest gas station" searches for a gas station near the car 2026-10-13 00:34:49 -07:00
Seb3thehacker 0bdc51ff6c voice: answer back when sworn at 2026-10-13 00:09:58 -07:00
Seb3thehacker 361caf5ea3 car audio: name the codec in the audio Setup, so cars that rejected it play music over USB 2026-10-13 00:09:27 -07:00
Seb3thehacker d53535e56c voice: download the models as plain files, skipping the slow bzip2 unpacking 2026-10-12 23:59:43 -07:00
Seb3thehacker 49a064acc0 video: cars that give no touchscreen size get the largest picture up to 1080p 2026-10-12 23:52:03 -07:00
Seb3thehacker 6d4389c7ab touch: shift taps past the car's cropped margin, so buttons land where they're pressed 2026-10-12 23:50:24 -07:00
Seb3thehacker 9fe463c4e7 Merge branch 'experimental' into wireless
# Conflicts:
#	gearslip/src/main/java/app/seb3thehacker/gearslip/ui/HomeScreen.kt
2026-10-12 22:09:43 -07:00
Seb3thehacker a3761534bb security: scan the Kotlin code with CodeQL by building the app 2026-10-12 21:10:36 -07:00
Seb3thehacker ceed61def9 security: scan the Kotlin code with CodeQL by building the app 2026-10-12 21:10:35 -07:00
Seb3thehacker fd0ef64a42 security: a weekly Trivy scan and Dependabot updates 2026-10-12 21:00:25 -07:00
Seb3thehacker 0d29263f0a security: a weekly Trivy scan and Dependabot updates 2026-10-12 21:00:20 -07:00
Seb3thehacker 28d99a3d87 voice: "play X" reopens the music app used last when none is open or it was closed 2026-10-12 17:16:25 -07:00
Seb3thehacker a526db6cd1 templated search: keep the keyboard down when the app asks, so results opened by voice show 2026-10-12 17:09:51 -07:00
Seb3thehacker 000ca85479 voice: Canary 180M as a middle choice for the final words 2026-10-12 16:38:32 -07:00
Seb3thehacker e76e9ddcf7 navigation: search near the phone, so "navigate to Del Taco" finds the closest one 2026-10-12 16:21:13 -07:00
Seb3thehacker 8069f54d8f voice: free Parakeet after 5 minutes without use and both models when the car disconnects 2026-10-12 16:20:15 -07:00
Seb3thehacker df85b0d1f3 car keyboard: taller keys, and search sits in the bottom-right corner 2026-10-12 16:19:27 -07:00
Seb3thehacker 4ac3d172d3 voice: understand natural phrasing for every command, open Gearslip's own apps by name, and say "exit" to hand the screen back 2026-10-12 16:10:23 -07:00
Seb3thehacker d099193ead car: call the Web app Browser 2026-10-12 16:02:53 -07:00
Seb3thehacker bc8f2372b1 voice: start listening sooner, say so while it gets ready, and scroll the help list from anywhere on it 2026-10-12 16:02:53 -07:00
Seb3thehacker 000bd1f3f4 bigger install image 2026-10-12 15:49:07 -07:00
Seb3thehacker 5fa9ed17e1 remove . 2026-10-12 15:44:21 -07:00
Seb3thehacker e2b9118354 readme: drop the link to the stats server's code 2026-10-12 15:43:21 -07:00
Seb3thehacker e1233cb1e4 readme: how Gearslip is made, a contributing section, and some badges 2026-10-12 15:40:56 -07:00
Seb3thehacker 3ca87d1afc log: explain that answering every car with protocol 1.3 is on purpose 2026-10-12 14:16:51 -07:00
Seb3thehacker 8e98851e47 0.3 intro: a recommended apps screen, and AntennaPod in the recommended list 2026-10-12 14:12:04 -07:00
Seb3thehacker af5308a9f9 readme: say plainly that AI writes nearly all the code 2026-10-12 12:44:26 -07:00
Seb3thehacker 7d6b9cf578 car audio: run the audio threads at audio priority so a busy phone doesn't make it skip 2026-10-12 01:31:24 -07:00
Seb3thehacker b2df84c46b voice and media search are built in now, with a voice intro screen and a new overlay over the car screen 2026-10-12 01:31:24 -07:00
Seb3thehacker 30d94bcfb5 voice: replace Whisper with sherpa-onnx, live words from Kroko and the final words from Parakeet
Models download in Settings instead of shipping in the APK. Credits for the engine and models are in About.
2026-10-12 00:51:31 -07:00
Seb3thehacker af9e9a7571 voice: play songs by name, a help list, and the phone's speech recognition as an option; Spotify connects without the wait 2026-10-12 00:20:36 -07:00
Seb3thehacker 7163598352 car media: drop the "Audio to car" line; name every audio focus state 2026-10-12 00:20:36 -07:00
Seb3thehacker 94ed58e13f 0.3 intro, donation links on Home, and About links for Matrix and the source 2026-10-12 00:20:36 -07:00
Seb3thehacker 59657c4f80 ixon fix mono color 2026-10-11 20:20:02 -07:00
Seb3thehacker 8abc8fe973 remove the "Before you plug in" popup on first launch 2026-10-11 20:01:36 -07:00
Seb3thehacker 4fe9421884 car audio: explain a blocked microphone permission instead of failing silently 2026-10-11 20:01:36 -07:00
Seb3thehacker 04b820cb22 wireless: wait for the car to connect when it publishes no Android Auto service 2026-10-11 17:22:58 -07:00
Seb3thehacker 669bf1f7ce Merge branch 'experimental' into wireless
# Conflicts:
#	gearslip/src/main/java/app/seb3thehacker/gearslip/GearslipActivity.kt
#	gearslip/src/main/java/app/seb3thehacker/gearslip/ui/HomeScreen.kt
2026-10-11 17:22:58 -07:00
Seb3thehacker 83916a6397 wireless connection attempt 2026-10-11 16:10:24 -07:00
Seb3thehacker 5a57608591 matrix link 2026-10-11 16:08:50 -07:00
Seb3thehacker f58ecf303a explain testing process and define AI use 2026-10-11 15:37:57 -07:00
Seb3thehacker 61a7188ad2 Add a contributor license agreement
Contributors keep their copyright and license their work to Twink Software LLC, which keeps it available under the AGPL and may also offer it under the commercial license. CONTRIBUTING now points to it.
2026-10-11 14:48:52 -07:00
Seb3thehacker fff2db63b0 updates: beta channel, and a switch to turn off automatic checks
Beta updates counts GitHub pre-releases as updates. With Check automatically off, Gearslip contacts GitHub only when you tap Check for updates.
2026-10-11 14:48:46 -07:00
Seb3thehacker fb6bc018d5 Add a contributor license agreement
Contributors keep their copyright and license their work to Twink Software LLC, which keeps it available under the AGPL and may also offer it under the commercial license. CONTRIBUTING now points to it.
2026-10-11 14:48:46 -07:00
Seb3thehacker 580a8365eb car screen: stop vibrating the phone on long presses 2026-10-11 11:39:22 -07:00
Seb3thehacker 5721b9efb3 icon: remove unused archive images 2026-10-11 11:34:06 -07:00
Seb3thehacker ed331088cc stats: name the phone's OS instead of only GrapheneOS or other
Reports GrapheneOS, CalyxOS, /e/OS, iodeOS, DivestOS, LineageOS, or stock with the brand.
2026-10-11 11:34:06 -07:00
Seb3thehacker 33ed2d0653 setup: drop the GrapheneOS exploit protection step 2026-10-11 11:34:06 -07:00
Seb3thehacker 261e6a719a car audio: choose Bluetooth or USB, offer Bluetooth after a refused screen share
USB audio needs Android's screen-sharing consent every drive. A new Car audio setting on the phone and car screens picks Bluetooth instead, and the first refusal opens a dialog offering Bluetooth, asking again, or the one-time adb command.
2026-10-11 11:34:06 -07:00
Seb3thehacker f9e38cbc45 icon: drop the black square behind the README icon 2026-10-11 00:59:45 -07:00
Seb3thehacker cfa01d4195 icon: drop the black square behind the README icon 2026-10-11 00:59:36 -07:00
Seb3thehacker e843f2adfe car knob: move focus with the rotary knob, show where it is
The knob's turns reached the car screen but never moved focus: keys handed
straight to the view left it in touch mode, where buttons refuse focus.
Buttons now show an accent ring while the knob is on them, map buttons wake
on a knob turn instead of only on touch, and web pages get the same ring.

The car's Home button opens the dashboard, Map opens the map, and Music
reopens the last media app.

Refs #9
2026-10-10 17:55:36 -07:00
Seb3thehacker 0af16d3348 Merge pull request #16 from iscle/fix/connection-handshake-lifecycle
Stop cleanly and ignore messages after handshake rejection
2026-10-10 14:47:46 -07:00
Iscle 8743e992d7 Stop cleanly and ignore messages after handshake rejection 2026-10-10 23:34:39 +02:00
Seb3thehacker 6f9fc061e3 tell the car about the Wi-Fi join before TCP 2026-10-09 15:30:41 -07:00
Seb3thehacker 239be1256b place pages: name on top, actions as buttons, call a place's number
Panes shaped like a place page (MAPS.ME's) show the name and kind first, short
tappable rows as buttons, and the details compact below. A phone number gets a
Call button that asks first with the dialer's confirmation. Other panes draw as
before.
2026-10-09 15:26:48 -07:00
Seb3thehacker 23f12884da car toasts: take turns instead of replacing each other
A new toast waits until the one on screen has had 2 seconds. Repeats restart the
visible toast, and a burst keeps only the newest three waiting.
2026-10-09 15:26:48 -07:00
Seb3thehacker 75dfaa3013 car icons: drop app tints that can't be seen, mark MAPS.ME and MapQuest as working
MAPS.ME tints its map buttons dark grey for dark mode, which vanished on Gearslip's
dark buttons. A tint below 3:1 contrast now falls back to the normal text colour,
as Android Auto does.
2026-10-09 15:26:48 -07:00
Seb3thehacker 179a51ea25 car toasts: match the message popup and stay up longer
A driver only glances at the screen, so short toasts now last 4 seconds and long
ones 7. The card uses the popup's colours, which read better over a map.
2026-10-09 14:24:32 -07:00
Seb3thehacker 006823232d Merge remote-tracking branch 'origin/experimental' into experimental 2026-10-09 14:24:28 -07:00
Seb3thehacker 0838bbb3cc Merge pull request #11 from iscle/feature/host-toasts
Show app toasts on the projected car display
2026-10-09 14:23:50 -07:00
Seb3thehacker 52cb769adc Merge pull request #10 from iscle/feature/host-media-category
Recognize media templates and group them under Music
2026-10-09 14:18:12 -07:00
Seb3thehacker 9c8b82d66a Merge branch 'experimental' into feature/host-media-category 2026-10-09 14:14:28 -07:00
Seb3thehacker 92d7940f56 car apps: let hosted apps start their own location service, fixing MapQuest
Since Android 14, an app bound by a visible host may start a foreground service
only if the host binds it with BIND_ALLOW_ACTIVITY_STARTS. Without it, MapQuest
crashed as navigation began, and Flitsmeister and Vela ran their services without
location. MapQuest now gets the working check.
2026-10-09 13:50:40 -07:00
Iscle 4c2c8c6dda Show app toasts on the projected car display 2026-10-09 20:58:12 +02:00
Iscle 4a6305052d Recognize media template apps and group them under Music 2026-10-09 20:50:43 +02:00
Seb3thehacker fd658d4e57 certificate: say where it comes from, offer import alongside download, keep cert files out of the repo 2026-10-08 15:27:48 -07:00
Seb3thehacker ddaf3e5657 move the certificate screen changes to the experimental branch 2026-10-08 15:27:36 -07:00
Seb3thehacker a549077c13 certificate: say where it comes from, offer import alongside download, keep cert files out of the repo 2026-10-08 15:25:04 -07:00
Seb3thehacker 246f571a86 readme: around 2020 2026-10-08 10:37:53 -07:00
Seb3thehacker a22d7936d4 readme: move which cars work to the top 2026-10-08 10:37:06 -07:00
Seb3thehacker 74330db01c readme: say most cars work, and why some newer ones don't 2026-10-08 10:35:30 -07:00
Seb3thehacker 516f6c7f94 ask once to share after the first drive that works, cache the cert for settings 2026-10-08 00:27:54 -07:00
Seb3thehacker 40086e061b audio pause when unplugged, theme tweak 2026-10-08 00:06:34 -07:00
Seb3thehacker c212c2e2d1 usage notes changes, audio changes, video quality increase. 2026-10-07 23:50:47 -07:00
Seb3thehacker fb4b0e2914 connection proxy negotiation 2026-10-07 23:41:16 -07:00
Seb3thehacker d0fd6ecb5c wireless crash fix 2026-10-07 22:40:41 -07:00
Seb3thehacker 39e1a5ee09 wireless func added (testing) 2026-10-07 18:45:57 -07:00
Seb3thehacker 55320c4d98 example data 2026-10-07 16:37:40 -07:00
Seb3thehacker 9dc705c8cf direct download link to obtainium 2026-10-07 15:11:06 -07:00
Seb3thehacker be981b0899 start car audio as soon as the head unit answers setup 2026-10-07 00:10:28 -07:00
Seb3thehacker dd3b845c7f stream directions to car directly 2026-10-06 23:33:18 -07:00
Seb3thehacker c57abecc62 send song data to car screens 2026-10-06 23:33:18 -07:00
Seb3thehacker dfebae93b3 higher video bitrate, scaled to the car's screen
800x480 at 30fps gets 8 Mbps instead of 4, and larger screens scale from
there. The encoder now declares the H.264 level each size needs, and the
bitrate stays within that level's limit.
2026-10-06 23:33:18 -07:00
Seb3thehacker 1af5eae8e7 usage notes switch on the Home screen 2026-10-06 23:33:18 -07:00
Seb3thehacker 25b8bbb1a5 keep the stats server out of the app repo 2026-10-06 23:33:18 -07:00
Seb3thehacker 2d5000e313 keep the stats server out of the app repo 2026-10-06 23:25:28 -07:00
Seb3thehacker 17534d5784 update checker: finds new releases, downloads and installs them
Checks GitHub's latest release once a day when the app opens and after a
drive. A new version shows as a card on Home, a notification after the
drive, and a card on the car screen at plug-in while parked. Downloads are
checked against GitHub's SHA-256 and never install while the car is
connected. Update notifications can be turned off in Settings.
2026-10-06 00:06:06 -07:00
Seb3thehacker 9620dab758 v0.2.00-alpha 2026-10-05 17:44:42 -07:00
Seb3thehacker e318b651bb leave the car's Bluetooth address out of logs 2026-10-05 17:44:42 -07:00
Seb3thehacker 3ac815eee4 setup guide shows what usage notes send 2026-10-05 17:44:42 -07:00
Seb3thehacker 10cc17bb66 usage notes preview shows the last cars sent 2026-10-05 17:44:42 -07:00
Seb3thehacker ca09d6adb8 usage notes say how they help fix cars 2026-10-05 17:44:42 -07:00
Seb3thehacker af058609dd dashboard shows GitHub's current download count 2026-10-05 17:44:42 -07:00
Seb3thehacker a4ab5b005e listen through the car's mic, fix exit on Uconnect, follow the car's night mode 2026-10-05 17:03:43 -07:00
Seb3thehacker 672a68cd1b ask for car audio on connect, media search behind experimental 2026-10-05 17:03:43 -07:00
Seb3thehacker 24d7e4c415 voice reply countdown keeps running through road noise 2026-10-05 17:03:43 -07:00
Seb3thehacker 1127d4baf7 phone typing bar sits on the keyboard, close on the left and search on the right 2026-10-05 17:03:43 -07:00
Seb3thehacker dc550ab735 fix the car screen falling behind on maps that draw nonstop 2026-10-05 17:03:43 -07:00
Seb3thehacker 5380cf0660 keep the car's drive safely button the width of its card 2026-10-05 01:47:39 -07:00
Seb3thehacker 7cc6b64ce2 ask existing users with the setup guide's usage page instead of a dialog 2026-10-05 01:39:41 -07:00
Seb3thehacker b5ff4ec22b send the first usage note after leaving the choices, not on the switch 2026-10-05 01:36:36 -07:00
Seb3thehacker 302dd6c344 show usage notes in dev builds too, but never send from them 2026-10-05 01:31:15 -07:00
Seb3thehacker 62fb8babad ask existing users about usage notes once 2026-10-05 01:27:49 -07:00
Seb3thehacker fe44576b06 dev builds send no usage notes 2026-10-05 01:24:49 -07:00
Seb3thehacker 355eb8bffd taller car keyboard, type from the phone, real media search
A box on the phone types into the car's text field. Media apps with library search show a list of results to pick from, searching a second after you stop typing.
2026-10-05 01:21:16 -07:00
Seb3thehacker 7d197bf021 fix silent audio on cars with an AAC sink, keep location out of logs
Cars like the 2022 Dacia Jogger list a compressed audio channel before the raw one, and Gearslip was sending raw audio to it. The Exit tile no longer shows as running, and the log leaves out the weather town and the car's GPS.
2026-10-05 01:21:16 -07:00
Seb3thehacker cf0c324d80 redesign the phone app's settings
One look for every section, What's new and the setup guide at the top, and the setup guide's buttons stay on screen. Recommended apps is hidden for now.
2026-10-05 01:21:16 -07:00
Seb3thehacker 5a3bbe82a0 opt-in usage notes and a stats dashboard
Release builds ask in setup and Settings whether to send a short note when the app is opened and after each drive: the version, plus the Android version, phone model and cars people tick. Nothing runs in the background. Dev builds always send. The Cloudflare Worker in stats-worker/ stores the notes and serves a private dashboard with GitHub downloads alongside.
2026-10-05 01:21:16 -07:00
Seb3thehacker b6bd6497b2 repeat the last video frame when the screen goes still
Decoders that show a frame only once the next arrives held the last change until
something else moved, so a closing menu looked a second late on LIVI.
2026-10-04 20:10:49 -07:00
Seb3thehacker 0eeb486d88 back buttons on every screen, exit to the car's own UI, smoother link
Every screen reached from Home has a themed back button in its header. A new Exit tile
asks the car to show its own interface, as Android Auto's does. All writes to the car
now go through one thread, so reading never waits on a write; LIVI no longer stalls
for two seconds at a time. Touches go to the front of the main thread's queue.
2026-10-04 19:57:43 -07:00
Seb3thehacker 1adeb6f7e1 wait for video acks instead of dropping frames
With one frame allowed in flight (LIVI), dropped frames forced keyframe repairs, which
caused more drops. LIVI went from 345 forced keyframes a minute to none.
2026-10-04 14:35:54 -07:00
Seb3thehacker bc0d6302d3 fix touch on head units with a second screen
LIVI lists an empty input for its cluster display before the real touchscreen, so every
tap was ignored. Pick the video and input that belong to the main screen.
2026-10-04 14:24:06 -07:00
Seb3thehacker 1341b4a08a fix LIVI not connecting
The head unit speaks first and gave up while the phone spent a second unlocking its
certificate before reading. Read straight away and unlock the certificate alongside it,
and keep it unlocked between connections.
2026-10-04 14:07:35 -07:00
Seb3thehacker c6b7f0e178 voice reply keeps listening for cancel, mic asked for in setup
The card now closes after no, cancel or no answer instead of popping back up. The
setup guide asks for the microphone even with car audio off, and its adb commands use
the installed package name. Sensor events are logged once per type plus a count.
2026-10-04 13:40:09 -07:00
Seb3thehacker d3592457cf hold marker in the theme engine, status badges on app icons
Any control with a press-and-hold action now shows it: a corner peel on Tactile and
Keycap that opens while held, a dot on Flat. Working/partial/broken badges moved onto
the app icon. Spotify and Deezer are marked partial since they need opening on the phone.
2026-10-04 13:36:11 -07:00
Seb3thehacker 024850bf48 fix car sensor channel, mark Metrolist working
Sensors live in service field 2, not 5 (that's the mic), so the channel never opened.
Message ids were also one off and the start request was missing min_update_period.
2026-10-02 15:51:15 -07:00
Seb3thehacker 96d16d048c reccomend apps to install during setup 2026-10-02 15:28:54 -07:00
Seb3thehacker 1b44d2d798 feature update and screenshots 2026-10-02 13:33:52 -07:00
Seb3thehacker e4725233dd Readme tagline update 2026-10-02 11:44:10 -07:00
Seb3thehacker 913e4af133 song search 2026-10-02 11:43:28 -07:00
Seb3thehacker 07d164ac74 voice reply improcement 2026-10-01 21:24:42 -07:00
Seb3thehacker 01a7ae1930 v0.1.10-alpha 2026-10-01 20:48:10 -07:00
Seb3thehacker c52a6c32b7 screen size auto negotiation 2026-10-01 20:43:26 -07:00
Seb3thehacker f21df41506 audi 32 to 64 bit fix 2026-10-01 20:37:04 -07:00
Seb3thehacker ff8a6a4cff app organization and movement options 2026-10-01 20:34:35 -07:00
Seb3thehacker a2d9ed7008 maps UI tweaks 2026-10-01 20:17:44 -07:00
Seb3thehacker 3994c9bd2e new supported apps, redesign UI for apps not fully supported 2026-10-01 14:28:44 -07:00
Seb3thehacker 4e4a246a51 voice prompt timer fixed 2026-10-01 12:55:40 -07:00
Seb3thehacker ebd0ba4add map screen wont rebuild on each app/page switch 2026-10-01 12:35:49 -07:00
Seb3thehacker 42785ce6be Merge pull request #8 from Brian5647/fix-map-freeze
fix the drawing after coming back from the media menu to navigation app
2026-10-01 12:19:40 -07:00
Seb3thehacker db205919c8 Point contributors at the experimental branch 2026-10-01 11:42:32 -07:00
Seb3thehacker f452099926 docs updates for developers 2026-10-01 11:33:27 -07:00
Seb3thehacker 03b74b1150 issue templates, doc updates 2026-10-01 11:20:14 -07:00
Brian5647 fbaeb0cc57 This is to fix the drawing after coming back from the media menu to the navigtion app. For organic maps the change from yesterday was fine. For flitsmeister it freezes the screen. 2026-10-01 19:03:45 +02:00
149 changed files with 12557 additions and 1554 deletions

No files matched your search

+15
View File
@@ -0,0 +1,15 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: Seb3thehacker
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: seb3thehacker
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']
+59
View File
@@ -0,0 +1,59 @@
name: Bug report
description: Something in Gearslip broke or behaves wrong.
labels: ["bug"]
body:
- type: markdown
attributes:
value: |
Thanks for reporting. Check the [open issues](https://github.com/twinksoftware/gearslip/issues) first, in case someone has already reported it.
- type: input
id: car
attributes:
label: Car
description: Make, model, and year.
placeholder: 2019 Honda Civic
validations:
required: true
- type: dropdown
id: head-unit
attributes:
label: Head unit
options:
- The one the car came with
- A factory unit from another car
- An aftermarket unit (Pioneer, Kenwood, Alpine, and so on)
- A homemade or open-source unit, such as LIVI
- I don't know
validations:
required: true
- type: dropdown
id: connection
attributes:
label: Connection
options:
- USB cable
- Wireless dongle
validations:
required: true
- type: textarea
id: what-happened
attributes:
label: What happened
description: What you did, what you expected, and what happened instead.
validations:
required: true
- type: textarea
id: log
attributes:
label: Log
description: |
On the phone, open Logs from Gearslip's home screen and tap Share log. Drag the file here.
The log records your phone, its Android version, the Gearslip version, and why the connection ended.
placeholder: Drop the log file here.
validations:
required: true
+65
View File
@@ -0,0 +1,65 @@
name: Car report
description: Tell us whether Gearslip works in your car, so others know before they install.
labels: ["car report"]
body:
- type: markdown
attributes:
value: |
Reports from working cars help as much as reports from broken ones.
- type: input
id: car
attributes:
label: Car
description: Make, model, and year.
placeholder: 2019 Honda Civic
validations:
required: true
- type: dropdown
id: head-unit
attributes:
label: Head unit
options:
- The one the car came with
- A factory unit from another car
- An aftermarket unit (Pioneer, Kenwood, Alpine, and so on)
- A homemade or open-source unit, such as LIVI
- I don't know
validations:
required: true
- type: dropdown
id: connection
attributes:
label: Connection
options:
- USB cable
- Wireless dongle
validations:
required: true
- type: dropdown
id: result
attributes:
label: Does it work?
options:
- Yes, everything I tried works
- Partly. It connects, but something is wrong
- No, it never connects
validations:
required: true
- type: textarea
id: notes
attributes:
label: Notes
description: What you tried, and anything that didn't work.
- type: textarea
id: log
attributes:
label: Log
description: |
Needed if the car doesn't work. On the phone, open Logs from Gearslip's home screen and tap Share log. Drag the file here.
placeholder: Drop the log file here.
+1
View File
@@ -0,0 +1 @@
blank_issues_enabled: false
@@ -0,0 +1,23 @@
name: Feature request
description: Suggest something Gearslip should do.
labels: ["enhancement"]
body:
- type: textarea
id: problem
attributes:
label: What do you want to do?
description: Describe the problem, not the solution. "I can't see my next turn while music plays" helps more than "add a turn widget."
validations:
required: true
- type: textarea
id: idea
attributes:
label: How should it work?
description: Your idea for a fix, if you have one.
- type: textarea
id: apps
attributes:
label: Apps involved
description: Any apps this touches, such as a map or music app.
+15
View File
@@ -0,0 +1,15 @@
# Weekly pull requests for newer Gradle libraries and GitHub Actions, aimed at experimental,
# where unreleased code lands. Security fixes come separately, as soon as an alert opens.
version: 2
updates:
- package-ecosystem: gradle
directory: /
target-branch: experimental
schedule:
interval: weekly
open-pull-requests-limit: 5
- package-ecosystem: github-actions
directory: /
target-branch: experimental
schedule:
interval: weekly
+15
View File
@@ -0,0 +1,15 @@
Gearslip 0.3.01 connects wirelessly on more phones and dongles.
## Fixed
- Wireless works with dongles that send their Wi-Fi details before their address.
- A VPN no longer stops a wireless drive without a word. Gearslip tries the phone's ordinary route around it, and if the VPN still blocks the car, Gearslip says so. To fix it for good, exclude Gearslip in your VPN app.
- When Android won't ask for the Nearby devices permission, a button opens Gearslip's settings, where you can turn it on.
## Changed
- An app that someone else has re-signed, such as a modified Waze, no longer shows the red X when it accepts Gearslip.
- Support Gearslip through Liberapay, Buy Me a Coffee, or Monero, in Settings.
- Gearslip now belongs to Twink Software LLC, and its code lives at github.com/twinksoftware/gearslip. Old links still work.
- The README lists Gearslip's signing certificate, so you can check your copy with AppVerifier.
## Known issues
- Phone calls are unstable right now and may drop or fail to connect.
+50
View File
@@ -0,0 +1,50 @@
name: CodeQL
# Kotlin can only be scanned while it compiles, so this builds the debug app, the same way
# release.yml does but without a signing key. Workflow files need no build.
on:
push:
branches: [main, experimental]
pull_request:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch:
jobs:
analyze:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
actions: read
strategy:
fail-fast: false
matrix:
include:
- language: actions
build-mode: none
- language: java-kotlin
build-mode: manual
steps:
- uses: actions/checkout@v4
- uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
- if: matrix.build-mode == 'manual'
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "21"
- if: matrix.build-mode == 'manual'
uses: gradle/actions/setup-gradle@v4
- if: matrix.build-mode == 'manual'
run: ./gradlew :gearslip:assembleDebug --no-daemon
- uses: github/codeql-action/analyze@v4
with:
category: "/language:${{ matrix.language }}"
+5
View File
@@ -45,4 +45,9 @@ jobs:
uses: softprops/action-gh-release@v2 uses: softprops/action-gh-release@v2
with: with:
files: ${{ steps.apk.outputs.path }} files: ${{ steps.apk.outputs.path }}
# The hand-written notes go first; GitHub adds the full changelog link under them.
body_path: .github/release-notes.md
generate_release_notes: true generate_release_notes: true
# Every release starts as a pre-release: only phones on the Alpha update channel get it.
# Once it's tested, untick pre-release on GitHub and everyone gets it.
prerelease: true
+34
View File
@@ -0,0 +1,34 @@
name: Security scan
# Trivy looks through the repository for committed secrets and risky workflow or build
# settings. Findings show under the Security tab, next to CodeQL's and Dependabot's.
on:
push:
branches: [main, experimental]
pull_request:
schedule:
- cron: "0 9 * * 1"
workflow_dispatch:
jobs:
trivy:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write
steps:
- uses: actions/checkout@v4
- uses: aquasecurity/trivy-action@v0.36.0
with:
scan-type: fs
scanners: vuln,secret,misconfig
format: sarif
output: trivy.sarif
severity: MEDIUM,HIGH,CRITICAL
- uses: github/codeql-action/upload-sarif@v4
if: always()
with:
sarif_file: trivy.sarif
category: trivy
+27 -17
View File
@@ -6,25 +6,35 @@
/build /build
local.properties local.properties
# Reference checkouts only - never referenced by the build; safe to delete entirely. # Keys
/cloned-repos
# The head unit certificate. Never committed or shared beyond who Seb sends it to directly.
/cert
# The release signing key and its passwords. Losing these breaks app updates; leaking them
# lets anyone sign an APK as Gearslip.
*.jks *.jks
*.keystore *.keystore
keystore.properties keystore.properties
/docs/cert-sweep.sh # Native build cache
archive/
CLAUDE.md
*.pdf
extraction/
.claude/
todo
logs/
# Native build cache.
.cxx/ .cxx/
# Testing
/cloned-repos
docs-internal/
# Internal
*.pdf
todo.md
logs/
# For AI code
.claude/
CLAUDE.md
# Other
archive/
.wrangler/
stats-worker/
# Certificates and keys never go in the repo; the app downloads or imports them on the phone.
*.p12
*.pfx
*.pem
*.crt
*.key
+49
View File
@@ -0,0 +1,49 @@
# Cars
Drivers who share usage notes tell us which cars Gearslip works in. This list comes from
those notes, and from our own testing. A car shows up here once Gearslip has reached its
screen at least once.
The year is the one the car reports. Some cars report the year of the head unit instead,
and some don't say.
## Works
| Car | Year | Head unit |
|---|---|---|
| Audi | | MIB2+ (Alpine) |
| Audi | | Harman |
| Chevrolet, with myChevrolet | 2019, 2020, 2021 | myChevrolet |
| Chevrolet, with MyLink | 2017 | MyLink |
| Dacia | | Media Nav (LG ULC 4.1) |
| Dacia | | LG ULC 4.5 |
| Ford | 2018 | SYNC 3.0 (Panasonic) |
| Ford Fiesta | 2019 | SYNC 3.2 (Panasonic) |
| Ford Fusion | 2018 | SYNC 3.1 (Panasonic) |
| Jeep Cherokee | 2019 | Uconnect 8.4 (Delphi) |
| MG | 2019 | Desay SV |
| Ram 1500 | 2018 | Uconnect 8.4 (Panasonic) |
| Škoda | 2018 | MIB Standard2 (TechniSat) |
| Subaru Outback | 2020 | Denso |
| Toyota RAV4 | 2020 | Pioneer T1 |
| Volkswagen | 2018 | MIB Standard2 (Delphi) |
| Volkswagen | 2024 | ICAS3 |
| Volvo XC90 | 2022 | SPA Plus (Mitsubishi Electric) |
| Wuling | 2021 | Desay SV |
| [LIVI](https://github.com/f-io/LIVI), the open-source head unit | | |
## Not working yet
| Car | Year | What happens |
|---|---|---|
| Honda | 2016 | The connection drops before Gearslip reaches the screen. |
| Honda | 2019 | The connection drops before Gearslip reaches the screen. |
| Mitsubishi Attrage | 2020 | The connection drops before Gearslip reaches the screen. |
Some cars reject Gearslip's certificate before they say what they are, so they can't appear
here by name. Most of these are from around 2020 or later.
## Add your car
Turn on usage notes in Gearslip's setup or Settings, and drive once. If your car is missing
or listed wrongly, open an issue.
+102
View File
@@ -0,0 +1,102 @@
# Gearslip Contributor License Agreement
Version 1.0, October 2026
Thank you for contributing to Gearslip. This agreement lets Twink Software LLC, the company
behind Gearslip, use your contribution. You keep the copyright in your work. You sign once,
and the agreement covers every contribution you make to Gearslip, past and future.
## Why we ask
Gearslip is free software under the GNU Affero General Public License, version 3 (the
"AGPL"). Twink Software LLC also sells commercial licenses to companies that cannot meet the
AGPL's terms, and that income helps pay for the project. Offering your code on those terms
needs your permission, and this agreement gives it.
## 1. Definitions
- **"We"** and **"us"** mean Twink Software LLC, a California limited liability company
(Secretary of State entity number B20260081700), under that name or any other, and its
successors and assigns.
- **"You"** means the person who signs this agreement.
- **"Contribution"** means any work of authorship, such as code, documentation, or images,
that you submit to Gearslip, whether before or after you sign. A submission includes a
pull request, patch, or issue attachment sent to a repository or channel we manage. It
excludes anything you mark in writing as "Not a Contribution".
## 2. Copyright license
You grant us, and anyone who receives software from us, a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable license to reproduce, prepare derivative
works of, publicly display, publicly perform, sublicense, and distribute your Contributions
and derivative works of them.
This license lets us offer your Contributions under licenses other than the AGPL, including
commercial licenses.
## 3. Patent license
You grant us, and anyone who receives software from us, a perpetual, worldwide,
non-exclusive, no-charge, royalty-free, irrevocable patent license to make, have made, use,
offer to sell, sell, import, and otherwise transfer your Contributions. This license covers
only the patent claims you can license that your Contribution, alone or combined with
Gearslip, would infringe.
If anyone sues claiming that Gearslip or a Contribution infringes a patent, the patent
licenses granted to that party for Gearslip end on the date the suit is filed.
## 4. Our promise to you
As long as we distribute a Contribution, we will also offer it under the AGPL or another
license approved by the Open Source Initiative. Your work will always stay available as
free software.
## 5. Your promises
You promise that:
1. **The work is yours.** Each Contribution is your original creation, and you have the
right to grant these licenses.
2. **Your employer agrees.** If your employer has rights in what you create, you have its
permission to contribute on its behalf, or it has waived its rights in your
Contributions.
3. **You name other people's work.** If any part of a Contribution is not your own, you say
so when you submit it, and you name its source and its license.
4. **You'll tell us of changes.** If any of these promises stops being true, you will tell
us.
## 6. No warranty
You provide your Contributions "as is", without warranties or conditions of any kind. You do
not have to provide support for them.
## 7. No obligation
We are not required to use or include any Contribution.
## 8. General
- California law governs this agreement.
- This agreement does not change the rights you have under the AGPL in Gearslip as
released.
- We may publish a new version of this agreement. A new version applies only to
Contributions you make after you sign it; this version keeps covering the ones you made
under it.
## Signing
Most contributors sign on GitHub: the CLA Assistant bot comments on your first pull request,
and you sign there with one click. Your GitHub account then serves as your signature.
To sign by hand instead, fill in the following and send it to us on Matrix. The address is
on the [Seb3thehacker GitHub profile](https://github.com/Seb3thehacker).
Name: ______________________________
GitHub username: ______________________________
Email: ______________________________
Signature: ______________________________
Date: ______________________________
+55
View File
@@ -0,0 +1,55 @@
# Contributing to Gearslip
Thanks for helping. Bug reports from real cars are the most useful thing you can send, so
start there if you aren't sure where to begin.
## Reporting a bug
Open an issue on [GitHub](https://github.com/twinksoftware/gearslip/issues) and include:
- your car's make, model, and year, and whether the head unit is the original one; and
- the log from the drive. Open Logs on the phone app's home screen and tap Share log.
The log records your phone, its Android version, the Gearslip version, the head unit, and
why the connection ended. Most car bugs can't be fixed without it. Head units often report
only a part number, so the log can't name your car for you.
## Building
You need Android Studio, or the Android SDK with JDK 21. Gradle downloads the NDK, CMake,
and the speech library on the first build.
```
./gradlew :gearslip:assembleDebug
```
The APK lands at `build/Gearslip-debug.apk`. It installs as Gearslip Dev, next to the
regular app.
## Pull requests
Work happens on the `experimental` branch, which merges into `main` at each release. Base your
branch on `experimental`, and open your pull request against it.
- Keep each pull request to one change, and explain the problem it solves.
- Say how you tested it: which car or head unit, which phone, and which apps.
- Code written with AI tools is welcome. Read it, test it, and stand behind it as you would
your own.
- Gearslip asks for no adb, root, or Shizuku. A feature must work with ordinary Android
permissions and on-device consent dialogs.
## Writing
Text that users read, such as screens, release notes, and docs, follows Strunk's
[*The Elements of Style*](https://www.gutenberg.org/ebooks/37134). In short: use the active
voice, state things positively, cut needless words, and use the serial comma.
## License
Gearslip is licensed under AGPL-3.0, and your contributions are too. Twink Software LLC, which
owns Gearslip, also sells a commercial license, described in [NOTICE.md](NOTICE.md).
Before we can merge your first pull request, you sign the
[Contributor License Agreement](CLA.md). You keep the copyright in your work, and it stays
available as free software; the agreement lets us offer it under the commercial license too.
A bot comments on your pull request with a link, and signing takes one click.
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (C) 2026 seb Copyright (C) 2026 Twink Software LLC and contributors
GNU AFFERO GENERAL PUBLIC LICENSE GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007 Version 3, 19 November 2007
+1 -1
View File
@@ -6,4 +6,4 @@ You may use, run, and modify Gearslip freely. If you distribute a modified versi
## Commercial licensing ## Commercial licensing
If the AGPL's terms don't work for you, the author sells a separate commercial license. The usual case is a company that wants to use Gearslip privately without publishing its changes. Terms, including any limits on modification, are set case by case. Contact me on Matrix; the address is on my profile. If the AGPL's terms don't work for you, Twink Software LLC, which owns Gearslip, sells a separate commercial license. The usual case is a company that wants to use Gearslip privately without publishing its changes. Terms, including any limits on modification, are set case by case. Contact us on Matrix; the address is on the [Seb3thehacker GitHub profile](https://github.com/Seb3thehacker).
+140 -25
View File
@@ -2,52 +2,149 @@
# Gearslip # Gearslip
Bring any app to your car screen. An Android Auto alternative built for everyone.
[![forthebadge](https://forthebadge.com/badges/compatibility-club-penguin.svg)](https://forthebadge.com)
[<img src="https://raw.githubusercontent.com/ImranR98/Obtainium/main/assets/graphics/badge_obtainium.png" alt="Get it on Obtainium" height="80">](https://apps.obtainium.imranr.dev/redirect?r=obtainium://add/https://github.com/twinksoftware/gearslip)
Questions, ideas?
[Join the discussion on Matrix](https://matrix.to/#/!mf0_mgBd5KqKpc9FfX_wHp1jFCcwAtPrndIn_oGIyMk?via=matrix.org).
## Which cars work
Most cars work. Starting around 2020, some cars added newer security standards that reject
Gearslip's certificate, and Gearslip can't run in those cars. [CARS.md](CARS.md) lists the
cars drivers have confirmed.
## What Gearslip is ## What Gearslip is
Gearslip is an Android Auto client for the phone. It speaks the Android Auto protocol Gearslip is an Android Auto client for the phone. It speaks the Android Auto protocol
directly to a car head unit over USB, with no Google software in the path. Gearslip acts directly to a car head unit, over USB or wirelessly, with no Google software in the path.
as a Car App Library host: an app that ships a car interface hands Gearslip a template, Gearslip acts as a Car App Library host: an app that ships a car interface hands Gearslip
and Gearslip renders that template natively on the car screen. Nothing is mirrored from a template, and Gearslip renders that template natively on the car screen. Nothing is
the phone. For apps without a car interface, Gearslip offers a separate screen-sharing mirrored from the phone. For apps without a car interface, Gearslip offers a separate
mode. screen-sharing mode.
| | |
|---|---|
| ![The launcher, with every car app on the phone and the route in the bar below](docs/screenshots/launcher.png) | ![Vela Maps showing the route, with the music player beside the map](docs/screenshots/navigation.png) |
| Your car apps in one place. A red X marks the ones that don't work yet. | Vela Maps navigating, with the player beside it. |
| ![ViVi Music's song, art, and controls in Gearslip's player](docs/screenshots/music.png) | ![A new message card over Organic Maps](docs/screenshots/message.png) |
| ViVi Music in Gearslip's player. | A new message, read aloud at a tap, over the map. |
| ![The dashboard, with the weather, the next calendar event, and recent notifications](docs/screenshots/dashboard.png) | ![DuckDuckGo open in Gearslip's web browser](docs/screenshots/web.png) |
| The weather, your next event, and what you missed. | Browse the web while parked. |
| ![The voice assistant listening over Organic Maps](docs/screenshots/assistant.png) | ![Coffee shops near you in Vela Maps, with the music player beside the map](docs/screenshots/places.png) |
| Ask for a place, a song, or an app. | Find coffee nearby without leaving the map. |
## Features ## Features
- **Phone calls.** Dial from a keypad, or search your contacts, then confirm before the - **Phone calls.** Dial from a keypad, or search your contacts, then confirm before the
call goes out. call goes out. When a call comes in, answer or decline it on the car screen.
- **Navigation.** Gearslip hosts real Car App Library map apps on the car screen; Organic - **Navigation.** Gearslip hosts real Car App Library map apps on the car screen.
Maps and OsmAnd both work today. Organic Maps, OsmAnd, and Flitsmeister work today. Vela runs, with rough
edges.
- **Music.** Browse, play, and queue tracks from a car-enabled media app, with a Now - **Music.** Browse, play, and queue tracks from a car-enabled media app, with a Now
Playing view, an Up Next queue, and lyrics. Playing view, an Up Next queue, and lyrics. Type a search, and apps that accept one
play the best match.
- **The web.** A browser template with its own on-screen keyboard, for anything that - **The web.** A browser template with its own on-screen keyboard, for anything that
doesn't ship a car interface of its own. doesn't ship a car interface of its own.
- **Weather and the day ahead.** A dashboard screen shows the week's calendar next to - **Weather and the day ahead.** A dashboard screen shows the week's calendar next to
the current and coming weather. the current and coming weather.
- **Notifications.** Read and reply to notifications from the car screen, using the same - **Voice.** Tap the mic, or press the talk button on the wheel, and say "navigate to the
on-screen keyboard. nearest gas station", "play Daft Punk", or "call Mom". Say "help" for the full list.
Speech turns into text on the phone, and nothing you say leaves it.
- **Messages.** New messages appear as a small card on the car screen. Gearslip reads
each one aloud, and you reply by voice or with a quick reply. Music pauses while
Gearslip speaks or listens. Speech runs on the phone, so it needs no Google account.
- **Notifications.** Only messages, missed calls, alarms, and reminders interrupt you.
The rest wait on the dashboard.
- **Your apps, your order.** The launcher groups apps by kind: phone, maps, music,
messaging, and the rest. Press and hold an app to move it.
- **Car controls.** Steering wheel buttons, control knobs, and D-pads all work.
- **Themes.** Choose flat buttons or skeuomorphic ones that look like real keys.
## Status ## Status
- **The connection is wired only, for now.** Plug the phone into the head unit's USB port; - **Wired or wireless.** Plug the phone into the car's USB port, or, in a car with wireless
Gearslip does not yet offer a wireless option. Android Auto, tap Connect wirelessly and pick the car. Pair the phone with the car over
Bluetooth first.
- **Setup walks you through it.** First launch asks for the Android Auto certificate, - **Setup walks you through it.** First launch asks for the Android Auto certificate,
then one permission at a time, each with a plain reason and an Allow or Skip. A help then one permission at a time, each with a plain reason and an Allow or Skip. A help
screen inside the car answers anything setup didn't cover. screen inside the car answers anything setup didn't cover.
- **Templated apps work.** Gearslip renders all 16 template types that the Car App - **Templated apps work.** Gearslip renders all 16 template types that the Car App
Library defines, and it draws its own keyboard for search, sign-in, and text fields. Library defines, and it draws its own keyboard for search, sign-in, and text fields.
It has been tested against several real navigation and media apps. It has been tested against several real navigation and media apps.
- **Gearslip works with [LIVI](https://github.com/f-io/LIVI),** LIVI, used in open source car dashboards, is fully supported by Gearslip. - **Gearslip works with [LIVI](https://github.com/f-io/LIVI),** the open-source head unit
- **Audio reaches the car by capturing playback, not by mirroring the screen.** Android used in homemade car dashboards.
requires a screen-share style consent dialog and indicator for this, even though - **Some apps refuse Gearslip.** Spotify, Waze, and Google Messages use a version of
Gearslip never reads the screen; that grant is the only way a non-privileged app can Google's car library that accepts only Android Auto itself. Spotify still plays through
tap another app's audio. The consent normally repeats every drive; two optional adb Gearslip's own player.
commands in the setup guide reduce it to a one-time grant. - **Calls use the car's Bluetooth,** as they do in Android Auto. Pair the phone with the
car to hear calls through its speakers.
- **Music reaches the car over Bluetooth or USB.** Setup asks which. Bluetooth is the
default: Android never asks for permission, and your map app's voice reaches the car
too. USB sounds a little clearer, with no delay, but Android shows a screen-share
consent dialog once per drive. Gearslip never reads the screen; capturing playback is
the only way an ordinary app can pass another app's audio along.
- **Gearslip needs no adb, root, or Shizuku.** Every feature relies on an ordinary - **Gearslip needs no adb, root, or Shizuku.** Every feature relies on an ordinary
Android permission or an on-device consent dialog. The two adb commands mentioned Android permission or an on-device consent dialog. For USB audio, setup offers two
above, for audio, are the sole exception: both are optional, and both only remove a optional adb commands: one removes the audio dialog for good, and the other keeps
dialog that the driver would otherwise see once per drive. notifications visible while audio plays.
- **Updates come from GitHub.** Once a day, Gearslip checks this repository's latest
release. When a new version is out, it downloads the APK, checks it against GitHub's
SHA-256, and installs it through Android's own installer. Turn off automatic checks in
Settings.
## Verify your download
Check that your copy of Gearslip is genuine with
[AppVerifier](https://github.com/soupslurpr/AppVerifier): share this block to it, or compare it
with the output of `apksigner verify --print-certs`.
```
app.seb3thehacker.gearslip
4C:C5:6E:BD:D1:3A:54:D2:60:EC:A9:B7:E2:B2:AE:6B:30:E2:F2:25:BF:DA:8F:50:8F:07:09:ED:F0:84:94:8B
```
## Usage notes
Gearslip can send short notes that show which cars it works in. They stay off unless you
turn them on, in setup or in Settings.
- **When they go out.** When you open Gearslip, at most once a day, and after each drive.
Nothing runs in the background.
- **What they carry.** The Gearslip version and which boxes you ticked, plus whatever
those boxes cover: the Android version and whether the phone runs GrapheneOS, the phone
model, whether a car connected that day, and each car's head unit, model, year, screen
size, and connection result, with whether that drive used the cable or wireless and
whether music went over USB or Bluetooth. If you tick it, how each drive went: how far
the connection got, how long it lasted, and how it ended. Settings shows the exact note. Untick a phone detail, and
the server drops the copy it kept.
- **What they leave out.** Your name, accounts, location, contacts, messages, and
anything you type or play.
- **How phones are counted.** A random number counts each phone once. The server keeps
only a salted hash of it.
- **Where they go.** To Gearslip's server. Like any website, it sees your IP address when a
note arrives, and it doesn't store it.
Turn the notes off, and they stop at once.
## How Gearslip is made
Gearslip is my beloved child. I decide what it does and how it looks, and Claude writes nearly
all of the code. I test every change myself before it ships: in a real car, with both a
GrapheneOS phone and a stock Android phone, and on [LIVI](https://github.com/f-io/LIVI), the
open-source head unit software. I watch the phone's temperature too, so a new feature doesn't
cook it. Then I beg my friends to try it in their cars, so it works in as many makes, models,
and years as possible. If something breaks, that's on me, so please open an issue.
## Contributing
Bug reports from real cars help most, and pull requests are welcome.
[CONTRIBUTING.md](CONTRIBUTING.md) explains what to put in a report, which branch to work
on, and the one-time agreement you sign before your first pull request merges.
## Building ## Building
@@ -67,7 +164,25 @@ keystore. Without them, the debug build still works.
[docs/BUILDING_APPS.md](docs/BUILDING_APPS.md) specifies how to write apps that Gearslip [docs/BUILDING_APPS.md](docs/BUILDING_APPS.md) specifies how to write apps that Gearslip
can render: templated apps, media apps, and the Gearslip app protocol. can render: templated apps, media apps, and the Gearslip app protocol.
## How Gearslip talks to cars
Gearslip is an independent program, written to work with car head units. What it knows of
the protocol comes from public open-source projects such as
[aasdk](https://github.com/opencardev/aasdk) and open-android-auto, and it contains none
of Google's Android Auto code.
Gearslip ships without a certificate. On first launch you choose where yours comes from:
download the one the aasdk project publishes, or import a file of your own. The
certificate stays on your phone. Gearslip never uploads, backs up, or logs it.
## Trademarks
Android, Android Auto, Google Messages, and Waze are trademarks of Google LLC. Spotify is
a trademark of Spotify AB. Gearslip is not affiliated with, endorsed by, or sponsored by
any of these companies. All other names belong to their owners.
## License ## License
Gearslip is licensed under AGPL-3.0; see [LICENSE](LICENSE). The author offers a separate Gearslip belongs to Twink Software LLC and is licensed under AGPL-3.0; see [LICENSE](LICENSE).
commercial license for private use outside the AGPL's terms; see [NOTICE.md](NOTICE.md). The LLC also sells a separate commercial license for private use outside the AGPL's terms; see
[NOTICE.md](NOTICE.md).
-38
View File
@@ -1,38 +0,0 @@
# Gearslip 0.1.0-alpha
This release lets you read and answer messages by voice, and the speech runs on your phone with no Google services.
## Messages and voice
- Messaging apps such as Signal and Molly open to your recent chats.
- Gearslip reads a message aloud, and you reply by voice. Say "cancel" or "change it" before it sends, or tap a quick reply such as your arrival time.
- Voice works on any phone, GrapheneOS included. Gearslip turns speech into text itself, so it needs no Google account.
- New messages show as a small card at the top right, even while you drive. A bar shows how long until the card closes or the reply sends.
- Only messages, missed calls, alarms, and reminders pop up. Everything else waits on the dashboard.
## Map
- The map stays loaded when you switch screens, so it no longer flashes black.
- Map apps draw more reliably and keep their connection when the phone is locked.
- The Home button shows your map app, so it no longer appears twice.
- The next turn shows in the nav bar when you leave the map.
## Controls and settings
- Steering wheel buttons, control knobs, and D-pads now work.
- To see the whole song title and artist in the nav bar, choose Settings > Player in the nav bar > Song title.
- Choose miles or kilometres under Settings > Units.
- "Button style" is now called "Theme".
- Try the voice assistant and the vehicle data screen under Settings > Experimental features.
- Logs now say why a car ended the connection.
## Known issues
- Spotify, Waze, and Google Messages updated to a version of Google's car library that only accepts Android Auto itself. Their own car screens won't open in Gearslip. Spotify still plays through the Gearslip player, which has the green check.
- Calls go through the car's Bluetooth, as they do in Android Auto. Pair your phone with the car to hear calls on its speakers.
## Install
Download `Gearslip-release.apk` below and open it on your phone. It updates 0.0.05-alpha in place and keeps your settings.
Found a bug, or a car where the buttons don't work? [Open an issue](https://github.com/Seb3thehacker/gearslip/issues) and share the log from the Logs screen in the phone app.
+65 -9
View File
@@ -1,11 +1,12 @@
# Building apps for Gearslip # Building apps for Gearslip
Gearslip renders three kinds of app on a car screen. This page specifies all three. Gearslip renders four kinds of app on a car screen. This page specifies all four.
| Kind | What it is | Status | | Kind | What it is | Status |
| --- | --- | --- | | --- | --- | --- |
| [Templated apps](#templated-apps) | An AndroidX Car App Library app. Gearslip hosts it. | Works today | | [Templated apps](#templated-apps) | An AndroidX Car App Library app. Gearslip hosts it. | Works today |
| [Media apps](#media-apps) | A `MediaBrowserService` that opted in to Android Auto. | Works today | | [Media apps](#media-apps) | A `MediaBrowserService` that opted in to Android Auto. | Works today |
| [Messaging apps](#messaging-apps) | An app whose notifications opted in to Android Auto. | Works today |
| [Gearslip apps](#gearslip-apps) | A native Gearslip screen, described over binder. | Specified here; not yet implemented | | [Gearslip apps](#gearslip-apps) | A native Gearslip screen, described over binder. | Specified here; not yet implemented |
Write a templated app when your app fits the Car App Library's templates, because it runs today Write a templated app when your app fits the Car App Library's templates, because it runs today
@@ -29,11 +30,11 @@ any other host.
Your app decides which hosts may render it, through the `HostValidator` your `CarAppService` Your app decides which hosts may render it, through the `HostValidator` your `CarAppService`
returns. returns.
Gearslip declares `android.car.permission.TEMPLATE_RENDERER`, and the library's own Gearslip declares `android.car.permission.TEMPLATE_RENDERER`. Car App Library 1.8.0-beta01
`HostValidator` accepts any host holding it. If you build your validator with and earlier, and 1.9.0-alpha01, accept any host holding it, so Gearslip passes the default
`HostValidator.Builder` and no allowlist, Gearslip already passes, and you need nothing below. validator. From 1.8.0-rc01 and 1.9.0-alpha02 the library checks that permission only on
Add an allowlist entry when you wrote a custom validator, or when you want to name Gearslip Android Automotive, so on a phone Gearslip passes only when your allowlist names it. Add this
explicitly: entry:
```xml ```xml
<!-- res/values/arrays.xml --> <!-- res/values/arrays.xml -->
@@ -143,7 +144,14 @@ existing video pipe. Gearslip never reads those pixels back.
Gearslip then reports which part of that surface the driver can see. Treat the visible area as Gearslip then reports which part of that surface the driver can see. Treat the visible area as
the region safe for your own controls, and the stable area as the region that never gets the region safe for your own controls, and the stable area as the region that never gets
covered. Gearslip keeps its chrome to a top bar, so both areas are the surface minus that bar. covered. Gearslip reports the same area for both, and it depends on the template:
| Template | What Gearslip covers |
| --- | --- |
| `NavigationTemplate` | A strip along the bottom for the trip estimate |
| `MapWithContentTemplate`, `MapTemplate`, and the place list and route preview templates | A content pane on the left |
The turn card and map buttons float over the visible area, as they do in Android Auto.
Your app receives four gestures, never raw touches: `onClick`, `onScroll`, `onFling` and Your app receives four gestures, never raw touches: `onClick`, `onScroll`, `onFling` and
`onScale`. Coordinates are surface pixels. Scroll distances follow `GestureDetector`'s `onScale`. Coordinates are surface pixels. Scroll distances follow `GestureDetector`'s
@@ -197,6 +205,53 @@ can start it playing on connection.
--- ---
## Messaging apps
Gearslip shows conversations from the notifications your app already posts, as Android Auto
does. Your app needs no car screen of its own.
Gearslip lists a messaging app when its car descriptor declares notifications:
```xml
<!-- res/xml/automotive_app_desc.xml -->
<automotiveApp>
<uses name="notification" />
</automotiveApp>
```
Reference it from the same `com.google.android.gms.car.application` meta-data that a media app
uses. Opening your tile on the car screen shows your recent conversations.
### Posting messages
Post each conversation as a `MessagingStyle` notification. Gearslip reads the conversation
title, each message's sender and text, and whether the chat is a group, and shows the last eight
messages. Leave the sender empty on the phone owner's own messages; Gearslip treats a message
with no sender as one the driver sent.
Gearslip reads new messages aloud and lets the driver reply by voice or with a quick reply. For
that, attach two actions, either to the notification or to its `WearableExtender`:
- **Reply:** an action carrying a `RemoteInput` that allows free-form input. Gearslip fills it
with the driver's reply and fires the action's intent.
- **Mark as read:** an action with `SEMANTIC_ACTION_MARK_AS_READ`. Gearslip shows a Mark read
button when you provide it.
### Which notifications interrupt the driver
Gearslip pops up a card for a conversation, a notification with a reply action, or a
notification in the `msg`, `missed_call`, `alarm`, `reminder` or `event` category. A channel
below default importance never pops up. Every other notification waits quietly on the
dashboard.
Gearslip drops these altogether:
- ongoing notifications and group summaries;
- the `transport`, `progress`, `service`, `sys` and `navigation` categories; and
- media notifications, because the player already shows them.
---
## Gearslip apps ## Gearslip apps
This section specifies protocol version 1. Gearslip does not yet implement it. This section specifies protocol version 1. Gearslip does not yet implement it.
@@ -376,7 +431,7 @@ long strings. Send a screen that fits, rather than one Gearslip has to cut.
"densityDpi": 160, "densityDpi": 160,
"batteryPercent": 82, "batteryPercent": 82,
"charging": true, "charging": true,
"vehicle": "Uconnect" "vehicle": "Honda"
} }
``` ```
@@ -416,6 +471,7 @@ home-screen pane comes later. Anything that needs its own rendering belongs in a
## Further reading ## Further reading
- [README.md](../README.md) - what Gearslip is and how to build it. - [README.md](../README.md) - what Gearslip is.
- [CONTRIBUTING.md](../CONTRIBUTING.md) - how to build Gearslip and report bugs.
- [Android for Cars App Library](https://developer.android.com/training/cars/apps) - the - [Android for Cars App Library](https://developer.android.com/training/cars/apps) - the
reference for templated apps. reference for templated apps.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

@@ -1,14 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" width="432" height="432" viewBox="0 0 108 108">
<defs>
<linearGradient id="trail" x1="73.4" y1="46.0" x2="95.8" y2="44.5" gradientUnits="userSpaceOnUse">
<stop offset="0%" stop-color="#8C1F1F" stop-opacity="0"/>
<stop offset="100%" stop-color="#8C1F1F" stop-opacity="0.7"/>
</linearGradient>
</defs>
<rect width="108" height="108" fill="#0c0c0d"/>
<circle cx="54" cy="54" r="52" fill="#131415"/>
<path d="M 73.23,47.15 L 94.77,51.90 L 96.89,37.05 L 73.57,44.78 Z" fill="url(#trail)"/>
<path d="M 54.00,27.00 L 59.99,19.91 Q 61.02,18.69 62.50,19.30 L 72.52,23.45 Q 74.00,24.07 73.87,25.66 L 73.23,33.31 Q 73.09,34.91 73.70,36.39 L 80.39,52.52 Q 81.00,54.00 82.22,55.03 L 88.09,59.99 Q 89.31,61.02 88.70,62.50 L 84.55,72.52 Q 83.93,74.00 82.34,73.87 L 73.09,73.09 Q 73.09,73.09 73.09,73.09 L 73.09,73.09 Q 73.09,73.09 73.09,73.09 L 73.87,82.34 Q 74.00,83.93 72.52,84.55 L 62.50,88.70 Q 61.02,89.31 59.99,88.09 L 54.00,81.00 Q 54.00,81.00 54.00,81.00 L 54.00,81.00 Q 54.00,81.00 54.00,81.00 L 48.01,88.09 Q 46.98,89.31 45.50,88.70 L 35.48,84.55 Q 34.00,83.93 34.13,82.34 L 34.91,73.09 Q 34.91,73.09 34.91,73.09 L 34.91,73.09 Q 34.91,73.09 34.91,73.09 L 25.66,73.87 Q 24.07,74.00 23.45,72.52 L 19.30,62.50 Q 18.69,61.02 19.91,59.99 L 27.00,54.00 Q 27.00,54.00 27.00,54.00 L 27.00,54.00 Q 27.00,54.00 27.00,54.00 L 19.91,48.01 Q 18.69,46.98 19.30,45.50 L 23.45,35.48 Q 24.07,34.00 25.66,34.13 L 34.91,34.91 Q 34.91,34.91 34.91,34.91 L 34.91,34.91 Q 34.91,34.91 34.91,34.91 L 34.13,25.66 Q 34.00,24.07 35.48,23.45 L 45.50,19.30 Q 46.98,18.69 48.01,19.91 L 54.00,27.00 Q 54.00,27.00 54.00,27.00 L 54.00,27.00 Q 54.00,27.00 54.00,27.00 Z" fill="#8B9096"/>
<path d="M 92.78,34.42 L 99.94,37.20 Q 101.43,37.78 101.34,39.37 L 100.67,50.20 Q 100.58,51.80 99.03,52.19 L 91.58,54.08 Q 90.03,54.47 90.13,52.87 L 91.19,35.44 Q 91.29,33.84 92.78,34.42 Z" fill="#8C1F1F"/>
<circle cx="54" cy="54" r="12" fill="#0c0c0d"/>
</svg>

Before

Width:  |  Height:  |  Size: 1.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 10 KiB

-1
View File
@@ -5,7 +5,6 @@
<stop offset="100%" stop-color="#8C1F1F" stop-opacity="0.7"/> <stop offset="100%" stop-color="#8C1F1F" stop-opacity="0.7"/>
</linearGradient> </linearGradient>
</defs> </defs>
<rect width="108" height="108" fill="#0c0c0d"/>
<circle cx="54" cy="54" r="52" fill="#131415"/> <circle cx="54" cy="54" r="52" fill="#131415"/>
<path d="M 73.23,47.15 L 94.77,51.90 L 96.89,37.05 L 73.57,44.78 Z" fill="url(#trail)"/> <path d="M 73.23,47.15 L 94.77,51.90 L 96.89,37.05 L 73.57,44.78 Z" fill="url(#trail)"/>
<path d="M 54.00,27.00 L 61.02,18.69 L 74.00,24.07 L 73.09,34.91 L 81.00,54.00 L 89.31,61.02 L 83.93,74.00 L 73.09,73.09 L 73.09,73.09 L 74.00,83.93 L 61.02,89.31 L 54.00,81.00 L 54.00,81.00 L 46.98,89.31 L 34.00,83.93 L 34.91,73.09 L 34.91,73.09 L 24.07,74.00 L 18.69,61.02 L 27.00,54.00 L 27.00,54.00 L 18.69,46.98 L 24.07,34.00 L 34.91,34.91 L 34.91,34.91 L 34.00,24.07 L 46.98,18.69 L 54.00,27.00 Z" fill="#8B9096"/> <path d="M 54.00,27.00 L 61.02,18.69 L 74.00,24.07 L 73.09,34.91 L 81.00,54.00 L 89.31,61.02 L 83.93,74.00 L 73.09,73.09 L 73.09,73.09 L 74.00,83.93 L 61.02,89.31 L 54.00,81.00 L 54.00,81.00 L 46.98,89.31 L 34.00,83.93 L 34.91,73.09 L 34.91,73.09 L 24.07,74.00 L 18.69,61.02 L 27.00,54.00 L 27.00,54.00 L 18.69,46.98 L 24.07,34.00 L 34.91,34.91 L 34.91,34.91 L 34.00,24.07 L 46.98,18.69 L 54.00,27.00 Z" fill="#8B9096"/>

Before

Width:  |  Height:  |  Size: 1.1 KiB

After

Width:  |  Height:  |  Size: 1.0 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 79 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 421 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 157 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 357 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 165 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 538 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 569 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 87 KiB

+25 -22
View File
@@ -24,6 +24,12 @@ val keystoreProperties = Properties().apply {
).firstOrNull { it.exists() }?.inputStream()?.use { load(it) } ).firstOrNull { it.exists() }?.inputStream()?.use { load(it) }
} }
// Keeps build output out of the module folder: <root>/build/gearslip/... instead of
// <root>/gearslip/build/..., so everything this project produces lands under one
// root-level build/ directory. Set before android {}, which works out the path to R8's default
// rules from it.
layout.buildDirectory.set(rootProject.layout.buildDirectory.dir("gearslip"))
android { android {
namespace = "app.seb3thehacker.gearslip" namespace = "app.seb3thehacker.gearslip"
compileSdk = 37 compileSdk = 37
@@ -32,9 +38,13 @@ android {
applicationId = "app.seb3thehacker.gearslip" applicationId = "app.seb3thehacker.gearslip"
minSdk = 31 minSdk = 31
targetSdk = 37 targetSdk = 37
versionCode = 6 versionCode = 10
versionName = "0.1.0-alpha" // -PgearslipVersionName=0.1.99-test makes a build that thinks it's older, to try the updater.
versionName = project.findProperty("gearslipVersionName")?.toString() ?: "0.3.01-alpha"
manifestPlaceholders["appLabel"] = "Gearslip" manifestPlaceholders["appLabel"] = "Gearslip"
// Where opted-in phones send their daily usage note: the stats-worker/ Worker's address,
// set as gearslipStatsUrl in gradle.properties. Left empty, nothing is ever sent.
buildConfigField("String", "STATS_URL", "\"${project.findProperty("gearslipStatsUrl") ?: ""}\"")
} }
signingConfigs { signingConfigs {
@@ -53,14 +63,23 @@ android {
buildTypes { buildTypes {
release { release {
signingConfig = signingConfigs.findByName("release") signingConfig = signingConfigs.findByName("release")
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
} }
// Its own package, so a dev build installs beside the release one instead of over it. // Its own package, so a debug build installs beside the release one instead of over it.
// Signed with the release key when it's here: both builds declare the car template // Signed with the release key when it's here: both builds declare the car template
// permission, and Android refuses a second app declaring it under a different key. // permission, and Android refuses a second app declaring it under a different key.
debug { debug {
applicationIdSuffix = ".dev" applicationIdSuffix = ".dev"
signingConfigs.findByName("release")?.let { signingConfig = it } signingConfigs.findByName("release")?.let { signingConfig = it }
manifestPlaceholders["appLabel"] = "Gearslip Dev" manifestPlaceholders["appLabel"] = "Gearslip Debug"
// -Pr8 shrinks a debug build like a release one, to test R8 in the debug install.
if (project.hasProperty("r8")) {
isMinifyEnabled = true
isShrinkResources = true
proguardFiles(getDefaultProguardFile("proguard-android-optimize.txt"), "proguard-rules.pro")
}
} }
} }
@@ -68,32 +87,16 @@ android {
buildConfig = true buildConfig = true
} }
// Keeps build output out of the module folder: <root>/build/gearslip/... instead of
// <root>/gearslip/build/..., so everything this project produces lands under one
// root-level build/ directory.
layout.buildDirectory.set(rootProject.layout.buildDirectory.dir("gearslip"))
compileOptions { compileOptions {
sourceCompatibility = JavaVersion.VERSION_21 sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21 targetCompatibility = JavaVersion.VERSION_21
} }
defaultConfig { defaultConfig {
// Speech-to-text ships for 64-bit ARM only: every phone Gearslip runs on, and one copy // Speech-to-text (sherpa-onnx, in jniLibs) ships for 64-bit ARM only: every phone
// of the native library instead of four. // Gearslip runs on, and one copy of the native library instead of four.
ndk { abiFilters += "arm64-v8a" } ndk { abiFilters += "arm64-v8a" }
} }
ndkVersion = "27.1.12297006"
externalNativeBuild {
cmake {
path = file("src/main/cpp/CMakeLists.txt")
version = "3.22.1"
}
}
androidResources {
// Model weights barely compress, and stored plainly they stream straight out of the APK.
noCompress += "bin"
}
testOptions { testOptions {
// GearslipLog calls android.util.Log, which is a stub on the local JVM. // GearslipLog calls android.util.Log, which is a stub on the local JVM.
+13
View File
@@ -0,0 +1,13 @@
# R8 shrinks the release build; these keep what it can't see being used.
# Names stay readable, so stack traces in shared logs need no mapping file. Shrinking does
# the real work on size.
-dontobfuscate
# Gearslip hosts other apps' car screens: their templates arrive as Bundles, and the car
# library rebuilds them by class name and reflection. Any model class an app might send has to
# survive, used by Gearslip's code or not.
-keep class androidx.car.app.** { *; }
# Speech-to-text: the native library calls back into these classes by name.
-keep class com.k2fsa.sherpa.onnx.** { *; }
+22
View File
@@ -4,6 +4,7 @@
<!-- The projected WebView loads real pages; without this every load fails silently. --> <!-- The projected WebView loads real pages; without this every load fails silently. -->
<uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />
<!-- <!--
Phone mirroring. Android refuses to hand over a MediaProjection unless a foreground Phone mirroring. Android refuses to hand over a MediaProjection unless a foreground
@@ -19,6 +20,22 @@
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" /> <uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION" /> <uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION" />
<!--
Wireless Android Auto: Bluetooth reaches the paired car's Android Auto service, which
hands over its Wi-Fi details; Gearslip then joins that network to reach the car.
-->
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT" />
<uses-permission android:name="android.permission.BLUETOOTH" android:maxSdkVersion="30" />
<uses-permission
android:name="android.permission.NEARBY_WIFI_DEVICES"
android:usesPermissionFlags="neverForLocation" />
<!-- Android 17 drops an app's traffic to the car's Wi-Fi without this. -->
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.CHANGE_NETWORK_STATE" />
<!-- The dashboard opened from the car clock: today's agenda and current weather. --> <!-- The dashboard opened from the car clock: today's agenda and current weather. -->
<uses-permission android:name="android.permission.READ_CALENDAR" /> <uses-permission android:name="android.permission.READ_CALENDAR" />
@@ -74,6 +91,10 @@
LAUNCHER query below to find, so the setup guide's Android Auto check needs this to see LAUNCHER query below to find, so the setup guide's Android Auto check needs this to see
whether the package is installed at all. --> whether the package is installed at all. -->
<package android:name="com.google.android.projection.gearhead" /> <package android:name="com.google.android.projection.gearhead" />
<!-- The opt-in usage note names the OS: GrapheneOS's setup app and /e/OS's app store give
them away (UsageStats.osName). -->
<package android:name="app.grapheneos.setupwizard" />
<package android:name="foundation.e.apps" />
<intent> <intent>
<action android:name="android.intent.action.MAIN" /> <action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" /> <category android:name="android.intent.category.LAUNCHER" />
@@ -123,6 +144,7 @@
android:name=".GearslipActivity" android:name=".GearslipActivity"
android:exported="true" android:exported="true"
android:launchMode="singleTop" android:launchMode="singleTop"
android:windowSoftInputMode="adjustResize"
android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboardHidden|uiMode|density|locale|layoutDirection|fontScale"> android:configChanges="orientation|screenSize|screenLayout|smallestScreenSize|keyboardHidden|uiMode|density|locale|layoutDirection|fontScale">
<intent-filter> <intent-filter>
-36
View File
@@ -1,36 +0,0 @@
# Speech-to-text for voice replies and the assistant: whisper.cpp, pinned to one commit and
# fetched at build time, so nothing outside this folder (cloned-repos included) is needed.
cmake_minimum_required(VERSION 3.22)
project(gearslip_speech CXX C)
set(CMAKE_CXX_STANDARD 17)
# Optimised in debug builds too. Unoptimised, Whisper runs several times slower, and a dev build
# would make voice look far worse than what ships.
foreach(lang C CXX)
set(CMAKE_${lang}_FLAGS_DEBUG "-O3 -DNDEBUG")
endforeach()
include(FetchContent)
FetchContent_Declare(
whisper
URL https://github.com/ggml-org/whisper.cpp/archive/6e4ab854f67f743900934a703d5603419384c961.tar.gz
URL_HASH SHA256=55b585aec5e1bc03e2a5cea61da97c6406fa833996cf7daeed704972273cc039
)
set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_TESTS OFF CACHE BOOL "" FORCE)
set(WHISPER_BUILD_SERVER OFF CACHE BOOL "" FORCE)
set(GGML_OPENMP OFF CACHE BOOL "" FORCE)
set(GGML_NATIVE OFF CACHE BOOL "" FORCE)
# ARMv8.2 with half floats and dot products: every phone that runs Android 12 or later in
# practice, and what makes Whisper fast enough to answer in under a second.
set(GGML_CPU_ARM_ARCH "armv8.2-a+dotprod+fp16" CACHE STRING "" FORCE)
FetchContent_MakeAvailable(whisper)
add_library(gearslip_speech SHARED speech_jni.cpp)
target_compile_options(gearslip_speech PRIVATE -O3)
find_library(LOG_LIB log)
target_link_libraries(gearslip_speech whisper android ${LOG_LIB})
target_link_options(gearslip_speech PRIVATE -Wl,--gc-sections -Wl,-z,max-page-size=16384)
-86
View File
@@ -1,86 +0,0 @@
// The bridge between Kotlin's SpeechEngine and whisper.cpp: load a model, turn one recorded
// phrase into text, free the model. Recording and deciding when a phrase ends stay in Kotlin.
#include <jni.h>
#include <android/asset_manager.h>
#include <android/asset_manager_jni.h>
#include <android/log.h>
#include <algorithm>
#include <cmath>
#include <string>
#include "whisper.h"
#define TAG "GearslipSpeech"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO, TAG, __VA_ARGS__)
static whisper_context_params context_params() {
whisper_context_params p = whisper_context_default_params();
p.use_gpu = false;
return p;
}
// Streams a model straight out of the APK, so the bundled one is never copied to storage.
static size_t asset_read(void *ctx, void *output, size_t size) {
int n = AAsset_read(static_cast<AAsset *>(ctx), output, size);
return n > 0 ? static_cast<size_t>(n) : 0;
}
static bool asset_eof(void *ctx) { return AAsset_getRemainingLength64(static_cast<AAsset *>(ctx)) <= 0; }
static void asset_close(void *ctx) { AAsset_close(static_cast<AAsset *>(ctx)); }
extern "C" JNIEXPORT jlong JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeInitAsset(JNIEnv *env, jobject, jobject assets, jstring jpath) {
AAssetManager *manager = AAssetManager_fromJava(env, assets);
const char *path = env->GetStringUTFChars(jpath, nullptr);
AAsset *asset = AAssetManager_open(manager, path, AASSET_MODE_STREAMING);
env->ReleaseStringUTFChars(jpath, path);
if (!asset) return 0;
whisper_model_loader loader = {asset, asset_read, asset_eof, asset_close};
return reinterpret_cast<jlong>(whisper_init_with_params(&loader, context_params()));
}
// Bytes, not a jstring: NewStringUTF wants modified UTF-8 and aborts on some of what Whisper can
// write (four-byte characters), so Kotlin decodes the plain UTF-8 instead.
extern "C" JNIEXPORT jbyteArray JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeTranscribe(
JNIEnv *env, jobject, jlong handle, jfloatArray jsamples, jstring jlang, jint threads) {
auto *ctx = reinterpret_cast<whisper_context *>(handle);
if (!ctx) return env->NewByteArray(0);
jsize n = env->GetArrayLength(jsamples);
jfloat *samples = env->GetFloatArrayElements(jsamples, nullptr);
const char *lang = env->GetStringUTFChars(jlang, nullptr);
whisper_full_params p = whisper_full_default_params(WHISPER_SAMPLING_GREEDY);
p.n_threads = threads;
p.language = lang;
p.translate = false;
p.no_context = true;
p.no_timestamps = true;
p.single_segment = true;
p.print_progress = false;
p.print_realtime = false;
p.print_special = false;
p.print_timestamps = false;
p.suppress_blank = true;
// The ACFT models are trained for a context cut to the audio's own length (1500 frames is
// 30 s, so 50 per second), which is what makes a three-second phrase fast. A little extra
// keeps the last word from being clipped.
p.audio_ctx = std::min(1500, static_cast<int>(std::ceil(n / 320.0)) + 32);
int rc = whisper_full(ctx, p, samples, n);
env->ReleaseStringUTFChars(jlang, lang);
env->ReleaseFloatArrayElements(jsamples, samples, JNI_ABORT);
if (rc != 0) {
LOGI("whisper_full failed: %d", rc);
return env->NewByteArray(0);
}
std::string text;
for (int i = 0; i < whisper_full_n_segments(ctx); i++) text += whisper_full_get_segment_text(ctx, i);
jbyteArray out = env->NewByteArray(static_cast<jsize>(text.size()));
env->SetByteArrayRegion(out, 0, static_cast<jsize>(text.size()), reinterpret_cast<const jbyte *>(text.data()));
return out;
}
extern "C" JNIEXPORT void JNICALL
Java_app_seb3thehacker_gearslip_speech_SpeechEngine_nativeFree(JNIEnv *, jobject, jlong handle) {
if (handle) whisper_free(reinterpret_cast<whisper_context *>(handle));
}
@@ -9,7 +9,6 @@ object AppSettings {
private const val PREFS = "gearslip_settings" private const val PREFS = "gearslip_settings"
private const val KEY_STARTUP_URL = "startup_url" private const val KEY_STARTUP_URL = "startup_url"
private const val KEY_SEEN_COMPAT_WARNING = "seen_compat_warning"
private const val KEY_SKIPPED_CERT_SETUP = "skipped_cert_setup" private const val KEY_SKIPPED_CERT_SETUP = "skipped_cert_setup"
private const val KEY_SEEN_PERMISSIONS_SETUP = "seen_permissions_setup" private const val KEY_SEEN_PERMISSIONS_SETUP = "seen_permissions_setup"
private const val KEY_GEARSLIP_ENABLED = "gearslip_enabled" private const val KEY_GEARSLIP_ENABLED = "gearslip_enabled"
@@ -32,14 +31,6 @@ object AppSettings {
prefs(context).edit().putString(KEY_STARTUP_URL, value.trim()).apply() prefs(context).edit().putString(KEY_STARTUP_URL, value.trim()).apply()
} }
/** Whether the newer-car compatibility warning has already been shown and dismissed once. */
fun hasSeenCompatWarning(context: Context): Boolean =
prefs(context).getBoolean(KEY_SEEN_COMPAT_WARNING, false)
fun setSeenCompatWarning(context: Context) {
prefs(context).edit().putBoolean(KEY_SEEN_COMPAT_WARNING, true).apply()
}
/** Whether the certificate setup screen has been deliberately skipped. */ /** Whether the certificate setup screen has been deliberately skipped. */
fun hasSkippedCertSetup(context: Context): Boolean = fun hasSkippedCertSetup(context: Context): Boolean =
prefs(context).getBoolean(KEY_SKIPPED_CERT_SETUP, false) prefs(context).getBoolean(KEY_SKIPPED_CERT_SETUP, false)
@@ -0,0 +1,28 @@
package app.seb3thehacker.gearslip
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* Who has the car's screen: Gearslip, or the car's own interface. Android Auto's "Exit" hands the
* screen back to the car with the session still running, so music keeps playing; the car's own
* Android Auto button brings Gearslip back. [GearslipRunner] fills in [request] while connected.
*/
object CarFocus {
private val projectedFlow = MutableStateFlow(true)
/** False while the car shows its own interface. */
val projected: StateFlow<Boolean> = projectedFlow
@Volatile internal var request: (() -> Unit)? = null
/** Asks the car to show its own interface. Does nothing when no car is connected. */
fun exitToCar() {
request?.invoke()
}
internal fun set(projected: Boolean) {
projectedFlow.value = projected
}
}
@@ -47,7 +47,10 @@ object CertProvider {
val kind: Kind = Kind.SELF_SIGNED, val kind: Kind = Kind.SELF_SIGNED,
) )
/** Loaded fresh each call, so a newly imported or pushed cert takes effect on the next connect. */ /**
* The identity for the next connection. A newly imported or pushed cert takes effect on the
* next connect: see [loadFromFiles] for how the cache notices.
*/
fun load(context: Context): Identity = loadFromFiles(context) ?: run { fun load(context: Context): Identity = loadFromFiles(context) ?: run {
val generated = SelfSignedCert.generate() val generated = SelfSignedCert.generate()
Identity( Identity(
@@ -65,7 +68,38 @@ object CertProvider {
*/ */
fun loadSupplied(context: Context): Identity? = loadFromFiles(context) fun loadSupplied(context: Context): Identity? = loadFromFiles(context)
/**
* Unlocking a PKCS#12 file takes about a second on a Pixel 6, which a head unit waiting for
* its first reply may not wait out. So the unlocked identity is kept, keyed on the files it
* could come from: importing, downloading, removing or pushing a cert changes the key.
*/
@Volatile private var cached: Pair<String, Identity>? = null
private fun sourcesKey(context: Context): String {
val dir = context.getExternalFilesDir(null)
val files = listOf(
importedFile(context, P12_NAME),
downloadedFile(context, P12_NAME),
dir?.let { File(it, P12_NAME) },
dir?.let { File(it, PASS_NAME) },
)
return files.joinToString("|") { f -> f?.let { "${it.path}:${it.lastModified()}:${it.length()}" } ?: "-" }
}
/** Unlocks the certificate ahead of time, so the first connection doesn't wait on it. */
fun warm(context: Context) {
runCatching { loadFromFiles(context) }
}
private fun loadFromFiles(context: Context): Identity? { private fun loadFromFiles(context: Context): Identity? {
val key = sourcesKey(context)
cached?.let { (k, identity) -> if (k == key) return identity }
val identity = unlockFromFiles(context) ?: return null
cached = key to identity
return identity
}
private fun unlockFromFiles(context: Context): Identity? {
val imported = importedFile(context, P12_NAME) val imported = importedFile(context, P12_NAME)
if (imported.isFile) { if (imported.isFile) {
runCatching { runCatching {
@@ -10,6 +10,64 @@ object Changelog {
class Release(val versionCode: Int, val versionName: String, val changes: List<String>) class Release(val versionCode: Int, val versionName: String, val changes: List<String>)
val releases = listOf( val releases = listOf(
Release(
9, "0.3.00-alpha",
listOf(
"Wireless Android Auto: pair the phone with the car once, and leave the cable at home.",
"Voice control is built in. Say \"navigate to a gas station\", \"play\" a song or artist, or open an app by name.",
"\"Navigate to\" picks the nearest place and stops at the map app's start screen, so you only tap Start.",
"Gearslip turns speech into text on the phone, with no Google account. Download the voice models once in setup.",
"The car's rotary knob moves between buttons, so cars without a touchscreen work too.",
"Choose how music reaches the car: Bluetooth, or USB with no delay. Gearslip offers to pair with the car the first time.",
"The car speaks each turn from your map app, and shows the directions on its own display where it can.",
"The car shows what's playing, and music apps get their own like, shuffle and repeat buttons.",
"Pick how the car's buttons look. Change it any time in the car's settings.",
"A new destination mid-route replaces the old route, and switching map apps ends the old app's route.",
"The car warns you when the phone's location is off.",
"Gearslip finds new versions and installs them for you.",
"More cars play music over USB, show a sharper picture, and put taps where you press.",
"On GrapheneOS, Gearslip no longer needs exploit protection compatibility mode.",
"The app is half the size it was.",
),
),
Release(
8, "0.2.00-alpha",
listOf(
"Exit takes you to the car's own screen. Tap Android Auto on the car to come back.",
"Gearslip listens through the car's microphone when the car has one, so voice works with the phone in a pocket.",
"A voice reply sends when its bar runs out, even over road noise or the radio. Say \"cancel\" or \"change it\" before then.",
"Maps turn dark when the car switches to night mode.",
"A moving map no longer makes the car screen fall behind your taps.",
"Gearslip asks to send sound to the car as soon as you connect, so videos in the web browser play in the car too.",
"Cars that offer more than one kind of music audio, such as the 2022 Dacia Jogger, play sound again.",
"When the car shows a keyboard, you can type on the phone instead. The car's keyboard is taller too.",
"Every car screen has a back button.",
"App icons show whether an app works in Gearslip. Metrolist works and shows the green check.",
"LIVI connects again, and touch works on cars with a second screen.",
"The phone's Settings screen has a new layout, with What's new and the setup guide at the top.",
"You can choose to send short usage notes that help the project. They hold no name, account, or location, and they're off unless you turn them on.",
"Searching inside a media app moved to Settings > Experimental features.",
"Under Settings > Media audio, the choices are now Phone audio and Car audio.",
"Logs leave out your location and the car's serial number.",
),
),
Release(
7, "0.1.10-alpha",
listOf(
"The launcher groups your apps: Phone, Maps, Music, Messaging, Web, Screen sharing, and Settings.",
"To move an app, press and hold it on the launcher, then tap Move left or Move right.",
"MapQuest and Flitsmeister work in Gearslip and show the green check.",
"Spotify has one tile. If Spotify isn't open on the phone, tap Open Spotify to start it.",
"Flick a map and it glides on, as it does in Android Auto.",
"The map picks up where it left off when you come back from another screen.",
"Search boxes in map apps are larger and easier to read.",
"Map apps keep the right icons on their buttons after they update.",
"When Gearslip asks \"Do you want to reply?\", a bar shows how long it will listen. The bar stops while you speak.",
"Gearslip fits its picture to wide car screens and keeps it clear of the edges the car crops.",
"Gearslip answers the car's connection checks correctly. Some cars, Audis among them, hung up after a second without this.",
"Google Maps no longer shows up as a messaging app.",
),
),
Release( Release(
6, "0.1.0-alpha", 6, "0.1.0-alpha",
listOf( listOf(
@@ -8,13 +8,17 @@ import android.hardware.usb.UsbAccessory
import android.hardware.usb.UsbManager import android.hardware.usb.UsbManager
import android.media.projection.MediaProjection import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager import android.media.projection.MediaProjectionManager
import android.net.Uri
import android.os.Bundle import android.os.Bundle
import android.os.ParcelFileDescriptor import android.os.ParcelFileDescriptor
import android.provider.Settings
import android.view.MotionEvent import android.view.MotionEvent
import android.view.Surface import android.view.Surface
import android.view.WindowManager import android.view.WindowManager
import androidx.activity.ComponentActivity import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent import androidx.activity.compose.setContent
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.activity.enableEdgeToEdge import androidx.activity.enableEdgeToEdge
import androidx.activity.result.contract.ActivityResultContracts.StartActivityForResult import androidx.activity.result.contract.ActivityResultContracts.StartActivityForResult
import android.content.res.Configuration import android.content.res.Configuration
@@ -27,6 +31,7 @@ import app.seb3thehacker.gearslip.car.CarEnvironment
import app.seb3thehacker.gearslip.car.CarSettings import app.seb3thehacker.gearslip.car.CarSettings
import app.seb3thehacker.gearslip.car.CarUi import app.seb3thehacker.gearslip.car.CarUi
import app.seb3thehacker.gearslip.car.Prefetch import app.seb3thehacker.gearslip.car.Prefetch
import app.seb3thehacker.gearslip.stats.UsageStats
import app.seb3thehacker.gearslip.ui.GearslipApp import app.seb3thehacker.gearslip.ui.GearslipApp
import app.seb3thehacker.gearslip.ui.GearslipTheme import app.seb3thehacker.gearslip.ui.GearslipTheme
import java.io.FileInputStream import java.io.FileInputStream
@@ -40,8 +45,12 @@ import java.io.FileOutputStream
* rotation or theme change from recreating it and dropping the connection. * rotation or theme change from recreating it and dropping the connection.
*/ */
class GearslipActivity : ComponentActivity(), Projection { class GearslipActivity : ComponentActivity(), Projection {
private val mainHandler = android.os.Handler(android.os.Looper.getMainLooper())
private var descriptor: ParcelFileDescriptor? = null private var descriptor: ParcelFileDescriptor? = null
/** The wireless session's socket, when the car came over Wi-Fi instead of USB. */
private var wirelessSocket: java.net.Socket? = null
private var runner: GearslipRunner? = null private var runner: GearslipRunner? = null
private var screenProjector: ScreenProjector? = null private var screenProjector: ScreenProjector? = null
private var worker: Thread? = null private var worker: Thread? = null
@@ -69,7 +78,13 @@ class GearslipActivity : ComponentActivity(), Projection {
private val audioPermission = registerForActivityResult( private val audioPermission = registerForActivityResult(
androidx.activity.result.contract.ActivityResultContracts.RequestPermission(), androidx.activity.result.contract.ActivityResultContracts.RequestPermission(),
) { granted -> ) { granted ->
if (granted) askAudioCapture() else { if (granted) askAudioCapture()
else if (!shouldShowRequestPermissionRationale(android.Manifest.permission.RECORD_AUDIO)) {
// Refused twice: Android now says no without showing anything, so the phone must.
GearslipLog.w("audio: record permission blocked - Android no longer asks")
CarAudio.micBlocked()
micBlockedNotice.value = true
} else {
GearslipLog.w("audio: record permission declined") GearslipLog.w("audio: record permission declined")
CarAudio.denied() CarAudio.denied()
} }
@@ -82,14 +97,22 @@ class GearslipActivity : ComponentActivity(), Projection {
} else { } else {
GearslipLog.w("audio: capture consent declined") GearslipLog.w("audio: capture consent declined")
CarAudio.denied() CarAudio.denied()
if (!CarSettings.audioChoiceAsked) offerBluetooth.value = true
} }
} }
/** Raised when the microphone permission is blocked; see [MicBlockedDialog]. */
private val micBlockedNotice = kotlinx.coroutines.flow.MutableStateFlow(false)
/** Raised once, after the first "no" to the capture dialog; see [AudioChoiceDialog]. */
private val offerBluetooth = kotlinx.coroutines.flow.MutableStateFlow(false)
private val detachReceiver = object : BroadcastReceiver() { private val detachReceiver = object : BroadcastReceiver() {
override fun onReceive(context: Context?, intent: Intent?) { override fun onReceive(context: Context?, intent: Intent?) {
if (intent?.action == UsbManager.ACTION_USB_ACCESSORY_DETACHED) { if (intent?.action == UsbManager.ACTION_USB_ACCESSORY_DETACHED) {
GearslipLog.w("accessory detached") GearslipLog.w("accessory detached")
closeAccessory() // A wireless session has no accessory to lose; leave it running.
if (wirelessSocket == null) closeAccessory()
} }
} }
} }
@@ -112,6 +135,9 @@ class GearslipActivity : ComponentActivity(), Projection {
GearslipLog.init(this) GearslipLog.init(this)
GearslipLog.installCrashHandler() GearslipLog.installCrashHandler()
GearslipLog.i("Gearslip ready") GearslipLog.i("Gearslip ready")
app.seb3thehacker.gearslip.wireless.WirelessLink.onConnected = { socket ->
runOnUiThread { startWireless(socket) }
}
CarSettings.init(this) CarSettings.init(this)
AppSettings.applyGearslipEnabled(this) AppSettings.applyGearslipEnabled(this)
// A capture that died without tidying up would leave notifications exposed to every // A capture that died without tidying up would leave notifications exposed to every
@@ -124,7 +150,33 @@ class GearslipActivity : ComponentActivity(), Projection {
setContent { setContent {
GearslipTheme { GearslipTheme {
GearslipApp(onDisconnect = ::closeAccessory, onRequestCallScreening = ::requestCallScreeningRole) GearslipApp(onDisconnect = { closeAccessory() }, onRequestCallScreening = ::requestCallScreeningRole)
val offer by offerBluetooth.collectAsState()
if (offer) {
fun close() {
CarSettings.markAudioChoiceAsked()
offerBluetooth.value = false
}
app.seb3thehacker.gearslip.ui.AudioChoiceDialog(
onUseBluetooth = {
CarSettings.setPipeAudio(false)
close()
},
onKeepAsking = ::close,
)
}
val micNotice by micBlockedNotice.collectAsState()
if (micNotice) {
app.seb3thehacker.gearslip.ui.MicBlockedDialog(
onOpenSettings = {
micBlockedNotice.value = false
startActivity(
Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS, Uri.fromParts("package", packageName, null)),
)
},
onDismiss = { micBlockedNotice.value = false },
)
}
} }
} }
@@ -148,6 +200,12 @@ class GearslipActivity : ComponentActivity(), Projection {
override fun onStart() { override fun onStart() {
super.onStart() super.onStart()
Prefetch.warm(this) Prefetch.warm(this)
UsageStats.onAppOpened(this)
app.seb3thehacker.gearslip.update.UpdateChecker.onAppOpened(this)
// Back from the app's settings with the microphone turned on: try car audio again.
if (checkSelfPermission(android.Manifest.permission.RECORD_AUDIO) == android.content.pm.PackageManager.PERMISSION_GRANTED) {
CarAudio.micAllowed()
}
} }
override fun onNewIntent(intent: Intent) { override fun onNewIntent(intent: Intent) {
@@ -157,6 +215,7 @@ class GearslipActivity : ComponentActivity(), Projection {
override fun onDestroy() { override fun onDestroy() {
super.onDestroy() super.onDestroy()
app.seb3thehacker.gearslip.wireless.WirelessLink.onConnected = null
runCatching { unregisterReceiver(detachReceiver) } runCatching { unregisterReceiver(detachReceiver) }
PhoneMirror.onStartRequested = null PhoneMirror.onStartRequested = null
PhoneMirror.onTokenReady = null PhoneMirror.onTokenReady = null
@@ -292,16 +351,44 @@ class GearslipActivity : ComponentActivity(), Projection {
worker = Thread { runner.run() }.apply { name = "gearslip"; start() } worker = Thread { runner.run() }.apply { name = "gearslip"; start() }
} }
/** A wireless session: the same runner, on the socket [WirelessLink] opened to the car. */
private fun startWireless(socket: java.net.Socket) {
closeAccessory(stopWireless = false)
if (!AppSettings.gearslipEnabled(this)) {
GearslipLog.i("wireless: Gearslip is switched off - not connecting")
runCatching { socket.close() }
return
}
GearslipLog.i("wireless: starting a session over Wi-Fi to ${socket.inetAddress?.hostAddress}:${socket.port}")
wirelessSocket = socket
val runner = GearslipRunner(
input = socket.getInputStream(),
output = socket.getOutputStream(),
identityProvider = { CertProvider.load(this) },
projection = this,
vehicleProfileFor = { info -> VehicleProfiles.find(VehicleProfiles.load(this), info) },
wireless = true,
)
this.runner = runner
worker = Thread { runner.run() }.apply { name = "gearslip"; start() }
}
private fun describe(accessory: UsbAccessory) { private fun describe(accessory: UsbAccessory) {
GearslipLog.i("accessory: manufacturer=${accessory.manufacturer} model=${accessory.model}") GearslipLog.i("accessory: manufacturer=${accessory.manufacturer} model=${accessory.model}")
GearslipLog.i(" version=${accessory.version} description=${accessory.description}") GearslipLog.i(" version=${accessory.version} description=${accessory.description}")
} }
private fun closeAccessory() { private fun closeAccessory() = closeAccessory(stopWireless = true)
/** Ends the session, wired or wireless. [stopWireless] also drops the car's Bluetooth and Wi-Fi. */
private fun closeAccessory(stopWireless: Boolean) {
runner?.stop() runner?.stop()
SessionReport.print() SessionReport.print()
runCatching { descriptor?.close() } runCatching { descriptor?.close() }
descriptor = null descriptor = null
runCatching { wirelessSocket?.close() }
wirelessSocket = null
if (stopWireless) app.seb3thehacker.gearslip.wireless.WirelessLink.stop(this)
runner = null runner = null
} }
@@ -323,7 +410,9 @@ class GearslipActivity : ComponentActivity(), Projection {
} }
override fun onTouch(action: Int, actionIndex: Int, points: List<TouchPoint>) { override fun onTouch(action: Int, actionIndex: Int, points: List<TouchPoint>) {
runOnUiThread { // To the front of the main thread's queue: a touch waiting behind other work is lag the
// driver feels, while the work it jumps ahead of can wait a frame.
mainHandler.postAtFrontOfQueue {
// Mirroring drives one phone touch at a time, so it follows the first finger only. // Mirroring drives one phone touch at a time, so it follows the first finger only.
if (PhoneMirror.active.value) points.firstOrNull()?.let { relayTouch(action, it.x, it.y) } if (PhoneMirror.active.value) points.firstOrNull()?.let { relayTouch(action, it.x, it.y) }
else screenProjector?.dispatchTouch(action, actionIndex, points) else screenProjector?.dispatchTouch(action, actionIndex, points)
@@ -331,6 +420,8 @@ class GearslipActivity : ComponentActivity(), Projection {
} }
override fun onProjectionStopped() { override fun onProjectionStopped() {
// Before the car UI goes, while the media connection is still up.
app.seb3thehacker.gearslip.car.CarServices.pausePhoneMedia()
runOnUiThread { runOnUiThread {
PhoneMirror.requestStop() PhoneMirror.requestStop()
app.seb3thehacker.gearslip.car.CarKeys.focusSink = null app.seb3thehacker.gearslip.car.CarKeys.focusSink = null
File diff suppressed because it is too large. Load diff
@@ -31,11 +31,17 @@ object Protobuf {
return varint(((field shl 3) or 2).toLong()) + varint(bytes.size.toLong()) + bytes return varint(((field shl 3) or 2).toLong()) + varint(bytes.size.toLong()) + bytes
} }
fun bytesField(field: Int, bytes: ByteArray): ByteArray =
varint(((field shl 3) or 2).toLong()) + varint(bytes.size.toLong()) + bytes
fun varintField(field: Int, value: Long): ByteArray = fun varintField(field: Int, value: Long): ByteArray =
varint((field shl 3).toLong()) + varint(value) varint((field shl 3).toLong()) + varint(value)
/** Reads the single int32 in AuthResponse. Negative values are 10-byte varints. */ /** Reads the single int32 in AuthResponse. Negative values are 10-byte varints. */
fun readInt32Field(data: ByteArray, field: Int): Int? { fun readInt32Field(data: ByteArray, field: Int): Int? = readInt64Field(data, field)?.toInt()
/** Reads a varint field whole, for int64s such as a ping's timestamp. */
fun readInt64Field(data: ByteArray, field: Int): Long? {
var pos = 0 var pos = 0
while (pos < data.size) { while (pos < data.size) {
val (tag, afterTag) = readVarint(data, pos) ?: return null val (tag, afterTag) = readVarint(data, pos) ?: return null
@@ -45,7 +51,7 @@ object Protobuf {
0 -> { 0 -> {
val (value, afterValue) = readVarint(data, pos) ?: return null val (value, afterValue) = readVarint(data, pos) ?: return null
pos = afterValue pos = afterValue
if (number == field) return value.toInt() if (number == field) return value
} }
2 -> { 2 -> {
val (len, afterLen) = readVarint(data, pos) ?: return null val (len, afterLen) = readVarint(data, pos) ?: return null
@@ -45,7 +45,19 @@ class ScreenProjector(private val context: Context) {
/** Tracks the synthetic gesture so injected MotionEvents have a coherent downTime. */ /** Tracks the synthetic gesture so injected MotionEvents have a coherent downTime. */
private var downTime = 0L private var downTime = 0L
fun start(surface: Surface, width: Int, height: Int, densityDpi: Int, content: @Composable () -> Unit) { /**
* [name] names the virtual display so the two callers are told apart from outside the app
* (`dumpsys display`): the real car session keeps "gearslip-projection", the on-phone bench
* preview passes "gearslip-preview".
*/
fun start(
surface: Surface,
width: Int,
height: Int,
densityDpi: Int,
name: String = "gearslip-projection",
content: @Composable () -> Unit,
) {
stop() stop()
val displayManager = context.getSystemService(DisplayManager::class.java) val displayManager = context.getSystemService(DisplayManager::class.java)
@@ -57,7 +69,7 @@ class ScreenProjector(private val context: Context) {
// driver on every connect. // driver on every connect.
val display = try { val display = try {
displayManager.createVirtualDisplay( displayManager.createVirtualDisplay(
"gearslip-projection", width, height, densityDpi, surface, name, width, height, densityDpi, surface,
DisplayManager.VIRTUAL_DISPLAY_FLAG_PRESENTATION or DisplayManager.VIRTUAL_DISPLAY_FLAG_PRESENTATION or
DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY, DisplayManager.VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY,
) )
@@ -186,6 +198,12 @@ class ScreenProjector(private val context: Context) {
*/ */
fun dispatchKey(keyCode: Int, down: Boolean) { fun dispatchKey(keyCode: Int, down: Boolean) {
val target = root ?: return val target = root ?: return
// Keys handed straight to the view skip the step where Android leaves touch mode, and in
// touch mode Compose's buttons refuse focus, so the knob had nothing to move between.
if (target.isInTouchMode) {
target.requestFocusFromTouch()
GearslipLog.i("keys: left touch mode for the knob/D-pad (now in touch mode: ${target.isInTouchMode})")
}
// Nothing on the display takes keys until something in it holds focus. // Nothing on the display takes keys until something in it holds focus.
if (target.findFocus() == null) target.requestFocus() if (target.findFocus() == null) target.requestFocus()
val shift = keyCode == app.seb3thehacker.gearslip.car.CarKeys.KEYCODE_SHIFT_TAB val shift = keyCode == app.seb3thehacker.gearslip.car.CarKeys.KEYCODE_SHIFT_TAB
@@ -20,8 +20,8 @@ object ServiceDiscovery {
/** /**
* [widthMargin]/[heightMargin] are the head unit's own numbers (VideoConfiguration fields 3 * [widthMargin]/[heightMargin] are the head unit's own numbers (VideoConfiguration fields 3
* and 4): pixels of the frame it does not treat as usable UI. Logged, and available to a * and 4): pixels of the frame it crops away, split evenly between the two edges, to fit the
* vehicle profile, but not applied automatically - what they mean varies by unit. * frame to a screen of a different shape. A matched vehicle profile's insets override them.
*/ */
class VideoConfig( class VideoConfig(
val resolution: Int, val resolution: Int,
@@ -38,11 +38,21 @@ object ServiceDiscovery {
fun pixelCount(): Int = pixelSize()?.let { it.first * it.second } ?: Int.MAX_VALUE fun pixelCount(): Int = pixelSize()?.let { it.first * it.second } ?: Int.MAX_VALUE
/** What's left on the car's screen once the margins are cropped off. */
fun visibleSize(): Pair<Int, Int>? = pixelSize()?.let { (w, h) -> (w - widthMargin) to (h - heightMargin) }
/** The margins as the strips of the frame the car UI keeps clear. */
fun marginInsets(): Insets = Insets(
top = heightMargin / 2, bottom = heightMargin - heightMargin / 2,
left = widthMargin / 2, right = widthMargin - widthMargin / 2,
)
override fun toString() = override fun toString() =
"$resolutionName @ $frameRateName, density=$density, margin=${widthMargin}x$heightMargin" "$resolutionName @ $frameRateName, density=$density, margin=${widthMargin}x$heightMargin"
} }
class VideoService(val serviceId: Int, val codecType: Int, val configs: List<VideoConfig>) { /** [displayId] is MediaSinkService field 6: 0, the default, is the main screen. */
class VideoService(val serviceId: Int, val codecType: Int, val configs: List<VideoConfig>, val displayId: Int = 0) {
val codecName: String get() = CODECS[codecType] ?: "unknown($codecType)" val codecName: String get() = CODECS[codecType] ?: "unknown($codecType)"
} }
@@ -89,6 +99,100 @@ object ServiceDiscovery {
} }
} }
/**
* The media sink to stream to. Gearslip sends raw PCM, so a PCM sink wins: the 2022 Dacia
* Jogger (LGE ULC 4.5) lists an AAC media sink on channel 3 before a PCM one on channel 5,
* and PCM pushed into the AAC sink was acked but played as silence. With no PCM sink, the
* first media sink is the best there is.
*/
fun pickMediaSink(sinks: List<AudioService>): AudioService? {
val media = sinks.filter { it.streamType == STREAM_MEDIA }
return media.firstOrNull { it.codecType == CODEC_PCM } ?: media.firstOrNull()
}
/**
* The channel the car takes "now playing" on (MediaPlaybackStatusService, Service field 9),
* which feeds its cluster and media screens. It carries no settings. Null when the car has none.
*/
fun findMediaStatusChannel(response: ByteArray): Int? = findChannelWith(response, 9)
/** The instrument cluster's turn-by-turn channel (NavigationStatusService, Service field 8). */
fun findNavStatusChannel(response: ByteArray): Int? = findChannelWith(response, 8)
private fun findChannelWith(response: ByteArray, serviceField: Int): Int? =
Wire.allBytes(Wire.fields(response), 1).firstNotNullOfOrNull { channel ->
val service = Wire.fields(channel)
if (service.any { it.number == serviceField }) Wire.varint(service, 1)?.toInt() else null
}
/**
* The car's microphone, from MediaSourceService (Service field 5):
* MediaSourceService { MediaCodecType available_type = 1; AudioConfiguration audio_config = 2;
* bool available_while_in_call = 3; }
* Null when the head unit has none.
*/
fun findMicService(response: ByteArray): AudioService? {
for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue
val source = Wire.fields(Wire.bytes(service, 5) ?: continue)
val f = Wire.fields(Wire.bytes(source, 2) ?: continue)
return AudioService(
serviceId = id,
streamType = 0,
codecType = Wire.varint(source, 1)?.toInt() ?: CODEC_PCM,
configs = listOf(
AudioConfig(
sampleRate = Wire.varint(f, 1)?.toInt() ?: 16_000,
bits = Wire.varint(f, 2)?.toInt() ?: 16,
channels = Wire.varint(f, 3)?.toInt() ?: 1,
),
),
)
}
return null
}
/**
* The car's Bluetooth, from BluetoothService (Service field 6):
* BluetoothService { string car_address = 1; repeated BluetoothPairingMethod methods = 2 [packed]; }
* Null when the car offers no pairing over the link.
*/
class BluetoothService(val serviceId: Int, val carAddress: String, val pairingMethods: List<Int>)
/** A carmaker's own channel (service field 12), for its apps on the phone: GM's is "com.gm.vip". */
class VendorService(val serviceId: Int, val name: String)
fun findVendorServices(response: ByteArray): List<VendorService> =
Wire.allBytes(Wire.fields(response), 1).mapNotNull { channel ->
val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: return@mapNotNull null
val fields = Wire.fields(Wire.bytes(service, 12) ?: return@mapNotNull null)
val name = Wire.bytes(fields, 1)?.let { String(it, Charsets.UTF_8) } ?: return@mapNotNull null
VendorService(id, name)
}
fun findBluetoothService(response: ByteArray): BluetoothService? {
for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue
val fields = Wire.fields(Wire.bytes(service, 6) ?: continue)
val address = Wire.bytes(fields, 1)?.let { String(it, Charsets.UTF_8) } ?: continue
// Packed as declared, though a car may send each method as its own varint.
val methods = fields.filter { it.number == 2 }.flatMap { f ->
f.bytes?.let(::packedVarints) ?: listOf(f.varint.toInt())
}
return BluetoothService(id, address, methods)
}
return null
}
const val STREAM_GUIDANCE = 1
const val STREAM_MEDIA = 3
const val CODEC_PCM = 1
const val CODEC_AAC_LC = 2
const val CODEC_AAC_LC_ADTS = 4
private val STREAMS = mapOf(0 to "NONE", 1 to "GUIDANCE", 2 to "SYSTEM_AUDIO", 3 to "MEDIA", 4 to "TELEPHONY") private val STREAMS = mapOf(0 to "NONE", 1 to "GUIDANCE", 2 to "SYSTEM_AUDIO", 3 to "MEDIA", 4 to "TELEPHONY")
private val RESOLUTIONS = mapOf( private val RESOLUTIONS = mapOf(
@@ -122,9 +226,23 @@ object ServiceDiscovery {
val touchWidth: Int, val touchWidth: Int,
val touchHeight: Int, val touchHeight: Int,
val keycodesSupported: List<Int> = emptyList(), val keycodesSupported: List<Int> = emptyList(),
/** InputSourceService field 5: which screen this input belongs to, 0 for the main one. */
val displayId: Int = 0,
) )
fun findInputService(response: ByteArray): InputService? { /**
* The input for the screen Gearslip projects to ([displayId]). A head unit with two screens
* offers an input service for each: LIVI lists an empty one for its second screen first,
* and taking that one left every touch on the real touchscreen ignored.
*/
fun findInputService(response: ByteArray, displayId: Int = 0): InputService? {
val inputs = inputServices(response)
return inputs.firstOrNull { it.displayId == displayId }
?: inputs.firstOrNull { it.touchWidth > 0 && it.touchHeight > 0 }
?: inputs.firstOrNull()
}
private fun inputServices(response: ByteArray): List<InputService> = buildList {
for (channel in Wire.allBytes(Wire.fields(response), 1)) { for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel) val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue val id = Wire.varint(service, 1)?.toInt() ?: continue
@@ -138,9 +256,8 @@ object ServiceDiscovery {
val keycodes = inputFields.filter { it.number == 1 }.flatMap { f -> val keycodes = inputFields.filter { it.number == 1 }.flatMap { f ->
f.bytes?.let(::packedVarints) ?: listOf(f.varint.toInt()) f.bytes?.let(::packedVarints) ?: listOf(f.varint.toInt())
} }
return InputService(id, width, height, keycodes) add(InputService(id, width, height, keycodes, Wire.varint(inputFields, 5)?.toInt() ?: 0))
} }
return null
} }
/** Decodes a `[packed=true]` repeated varint field's raw bytes into individual values. */ /** Decodes a `[packed=true]` repeated varint field's raw bytes into individual values. */
@@ -160,7 +277,16 @@ object ServiceDiscovery {
} }
} }
/**
* The main screen's video sink. A head unit with a second screen, such as LIVI with a
* cluster display, offers a sink for each; the one without a display id (0) is the main one.
*/
fun findVideoService(response: ByteArray): VideoService? { fun findVideoService(response: ByteArray): VideoService? {
val sinks = videoServices(response)
return sinks.firstOrNull { it.displayId == 0 } ?: sinks.firstOrNull()
}
private fun videoServices(response: ByteArray): List<VideoService> = buildList {
for (channel in Wire.allBytes(Wire.fields(response), 1)) { for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel) val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue val id = Wire.varint(service, 1)?.toInt() ?: continue
@@ -181,15 +307,14 @@ object ServiceDiscovery {
heightMargin = Wire.varint(f, 4)?.toInt() ?: 0, heightMargin = Wire.varint(f, 4)?.toInt() ?: 0,
) )
} }
return VideoService(id, codecType, configs) add(VideoService(id, codecType, configs, Wire.varint(sinkFields, 6)?.toInt() ?: 0))
} }
return null
} }
/** /**
* The head unit's sensor feed, from SensorSourceService (Service field 5): * The head unit's sensor feed, from SensorSourceService (Service field 2; field 5 is the car's mic):
* SensorSourceService { repeated SensorType sensors = 1; } * SensorSourceService { repeated Sensor sensors = 1; }
* SensorType { required int32 type = 1; } * Sensor { required SensorType sensor_type = 1; }
* *
* Field numbers for what actually rides inside each SensorEvent once subscribed are not * Field numbers for what actually rides inside each SensorEvent once subscribed are not
* pinned down here - unlike video/audio/input, there's no independent source to check them * pinned down here - unlike video/audio/input, there's no independent source to check them
@@ -201,7 +326,7 @@ object ServiceDiscovery {
for (channel in Wire.allBytes(Wire.fields(response), 1)) { for (channel in Wire.allBytes(Wire.fields(response), 1)) {
val service = Wire.fields(channel) val service = Wire.fields(channel)
val id = Wire.varint(service, 1)?.toInt() ?: continue val id = Wire.varint(service, 1)?.toInt() ?: continue
val sensorSource = Wire.bytes(service, 5) ?: continue val sensorSource = Wire.bytes(service, 2) ?: continue
val types = Wire.allBytes(Wire.fields(sensorSource), 1).mapNotNull { val types = Wire.allBytes(Wire.fields(sensorSource), 1).mapNotNull {
Wire.varint(Wire.fields(it), 1)?.toInt() Wire.varint(Wire.fields(it), 1)?.toInt()
} }
@@ -1,6 +1,7 @@
package app.seb3thehacker.gearslip package app.seb3thehacker.gearslip
import android.content.Context import android.content.Context
import app.seb3thehacker.gearslip.stats.UsageStats
import android.os.Build import android.os.Build
/** /**
@@ -28,12 +29,30 @@ object SessionReport {
UNKNOWN("unclassified"), UNKNOWN("unclassified"),
} }
/**
* How far a session got, in the order a connection is built. The usage note's "Where
* connections fail" share sends the furthest one reached, so a failure says where it broke.
*/
enum class Stage(val key: String) {
USB("usb"), // the link opened; the car may have said nothing
VERSIONS("versions"), // the car said which protocol it speaks
TLS("tls"), // the secure link is up
ACCEPTED("accepted"), // the car accepted the certificate
CHANNELS("channels"), // the car listed its channels and said who it is
VIDEO("video"), // the car's screen is set up for Gearslip
STREAMING("streaming"), // video frames are reaching the car
}
private var appVersion = "?" private var appVersion = "?"
private var appCode = 0L private var appCode = 0L
@Volatile private var startedAt = 0L @Volatile private var startedAt = 0L
@Volatile var category = Category.NONE; private set @Volatile var category = Category.NONE; private set
@Volatile private var failure = "" @Volatile private var failure = ""
@Volatile var stage = Stage.USB; private set
/** What the car or the phone said when it broke: a TLS alert, an auth status, a ByeBye reason, an errno. */
@Volatile private var failCode = ""
@Volatile private var failedAfter = 0L
@Volatile private var protocolVersion = "" @Volatile private var protocolVersion = ""
@Volatile private var certSource = "" @Volatile private var certSource = ""
@Volatile private var certIssuer = "" @Volatile private var certIssuer = ""
@@ -48,41 +67,114 @@ object SessionReport {
@Volatile private var lastState = "" @Volatile private var lastState = ""
private var printed = false private var printed = false
private var appContext: Context? = null
fun init(context: Context) { fun init(context: Context) {
appContext = context.applicationContext
val info = runCatching { context.packageManager.getPackageInfo(context.packageName, 0) }.getOrNull() val info = runCatching { context.packageManager.getPackageInfo(context.packageName, 0) }.getOrNull()
appVersion = info?.versionName ?: "?" appVersion = info?.versionName ?: "?"
appCode = info?.longVersionCode ?: 0 appCode = info?.longVersionCode ?: 0
} }
fun begin() { /** How this drive reached the car, and how its music did: "usb" or "wireless", "usb" or "bluetooth". */
@Volatile var link = ""; private set
@Volatile var audioMode = ""; private set
fun begin(wireless: Boolean) {
startedAt = System.currentTimeMillis() startedAt = System.currentTimeMillis()
link = if (wireless) "wireless" else "usb"
audioMode = if (app.seb3thehacker.gearslip.car.CarSettings.pipeAudio.value) "usb" else "bluetooth"
category = Category.NONE category = Category.NONE
failure = ""; protocolVersion = ""; certSource = ""; certIssuer = "" failure = ""; protocolVersion = ""; certSource = ""; certIssuer = ""
stage = Stage.USB; failCode = ""; failedAfter = 0
tlsProtocol = ""; tlsCipher = ""; authStatus = null tlsProtocol = ""; tlsCipher = ""; authStatus = null
headUnitInfo = null
carScreen = ""; carDpi = 0
headUnit = ""; profile = ""; video = ""; lastStream = ""; lastState = "" headUnit = ""; profile = ""; video = ""; lastStream = ""; lastState = ""
printed = false printed = false
} }
fun protocolVersion(major: Int, minor: Int) { protocolVersion = "$major.$minor" } /** Moves the session's furthest stage on; never back. */
private fun reached(next: Stage) { if (next > stage) stage = next }
fun protocolVersion(major: Int, minor: Int) { protocolVersion = "$major.$minor"; reached(Stage.VERSIONS) }
fun certificate(source: String, issuer: String) { certSource = source; certIssuer = issuer } fun certificate(source: String, issuer: String) { certSource = source; certIssuer = issuer }
fun tls(protocol: String, cipher: String) { tlsProtocol = protocol; tlsCipher = cipher } fun tls(protocol: String, cipher: String) { tlsProtocol = protocol; tlsCipher = cipher; reached(Stage.TLS) }
fun auth(status: Int) { authStatus = status } fun auth(status: Int) { authStatus = status; if (status == 0) reached(Stage.ACCEPTED) }
fun headUnit(info: String) { headUnit = info } fun headUnit(info: String) { headUnit = info }
/** Who the car said it is, for the opt-in usage note; null until service discovery. */
@Volatile var headUnitInfo: ServiceDiscovery.HeadUnitInfo? = null
set(value) { field = value; if (value != null) reached(Stage.CHANNELS) }
/** The car screen's usable size once its margins are cropped, and its density; blank before video starts. */
@Volatile var carScreen = ""; private set
@Volatile var carDpi = 0; private set
fun carScreen(size: String, dpi: Int) { carScreen = size; carDpi = dpi; reached(Stage.VIDEO) }
fun profile(name: String?) { profile = name ?: "none matched" } fun profile(name: String?) { profile = name ?: "none matched" }
fun video(summary: String) { video = summary } fun video(summary: String) { video = summary }
fun audio(summary: String) { audio = summary } fun audio(summary: String) { audio = summary }
fun stream(summary: String) { lastStream = summary } fun stream(summary: String) { lastStream = summary; reached(Stage.STREAMING) }
/** Records the first failure only: later ones are usually fallout from it. */ /**
fun fail(category: Category, detail: String, state: String = "") { * Records the first failure only: later ones are usually fallout from it. [code] is the short
* thing the car or the phone said, like "auth -3" or "bye 1", for the usage note.
*/
fun fail(category: Category, detail: String, state: String = "", code: String = "") {
if (this.category != Category.NONE) return if (this.category != Category.NONE) return
this.category = category this.category = category
failure = detail failure = detail
lastState = state lastState = state
failCode = code
failedAfter = if (startedAt == 0L) 0 else (System.currentTimeMillis() - startedAt) / 1000
} }
fun fail(category: Category, detail: String, t: Throwable, state: String = "") = fun fail(category: Category, detail: String, t: Throwable, state: String = "") =
fail(category, "$detail (${GearslipLog.rootCause(t)})", state) fail(category, "$detail (${GearslipLog.rootCause(t)})", state, codeOf(t))
/**
* A short code from an exception, never its whole message: the TLS alert's name, the errno
* of a USB failure, or else the exception's class.
*/
private fun codeOf(t: Throwable): String {
var root = t
while (root.cause != null && root.cause !== root) root = root.cause!!
val text = "${t.message} ${root.message}"
Regex("(?i)alert[ _:]*\\(?([a-z_ ]{3,40}?)\\)?(?:\\s|$|[,;.])").find(text)?.let {
return "alert ${it.groupValues[1].trim().lowercase().replace(' ', '_')}"
}
Regex("\\b(E[A-Z]{2,12})\\b").find(text)?.let { return it.groupValues[1] }
return root.javaClass.simpleName.take(40)
}
/**
* One session for the usage note's "Where connections fail" share: how far it got, how and
* when it ended, and with which kind of certificate. Counts and codes only, nothing personal.
*/
fun sessionNote(): org.json.JSONObject {
val seconds = if (startedAt == 0L) 0 else (System.currentTimeMillis() - startedAt) / 1000
val source = certSource.lowercase()
return org.json.JSONObject()
.put("stage", stage.key)
.put("ended", category.name.lowercase())
.put("code", failCode)
.put("failed_at", if (category == Category.NONE) 0 else rounded(failedAfter))
.put("seconds", rounded(seconds))
.put("cert", when {
"imported" in source -> "imported"
"downloaded" in source -> "downloaded"
"self" in source -> "self-signed"
source.isEmpty() -> ""
else -> "other"
})
}
/** Seconds rounded so a drive's length can't be matched to anything: 5 s, then minutes, then 5 minutes. */
private fun rounded(s: Long): Long = when {
s < 60 -> (s + 2) / 5 * 5
s < 3600 -> (s + 30) / 60 * 60
else -> (s + 150) / 300 * 300
}
/** The report as plain text, for the log file and for anything that gets sent. */ /** The report as plain text, for the log file and for anything that gets sent. */
fun render(): String { fun render(): String {
@@ -121,6 +213,25 @@ object SessionReport {
if (printed || startedAt == 0L) return if (printed || startedAt == 0L) return
printed = true printed = true
render().lines().forEach { GearslipLog.i(it) } render().lines().forEach { GearslipLog.i(it) }
appContext?.let { context ->
// Video started means the car showed Gearslip: that's a working car, and unplugging or
// switching the car off at the end shows up as a USB or ByeBye "failure" that isn't one.
val reached = carScreen.isNotEmpty()
val outcome = when {
reached && category in setOf(Category.NONE, Category.USB, Category.BYEBYE) -> "connected"
reached -> "connected, then ${category.label}"
category == Category.NONE -> "ended before video"
else -> category.label
}
runCatching {
UsageStats.recordSession(context, headUnitInfo, protocolVersion, outcome, carScreen, carDpi, sessionNote(), link, audioMode)
}
if (outcome == "connected") runCatching {
app.seb3thehacker.gearslip.stats.WorkedPrompt.onDriveWorked(context, headUnitInfo, protocolVersion, carScreen, carDpi)
app.seb3thehacker.gearslip.ui.DonatePrompt.onDriveWorked(context)
}
runCatching { app.seb3thehacker.gearslip.update.UpdateChecker.onDriveEnded(context) }
}
GearslipLog.flush() GearslipLog.flush()
} }
@@ -20,7 +20,7 @@ object SessionStatus {
Snapshot( Snapshot(
Phase.IDLE, Phase.IDLE,
"Not connected", "Not connected",
"Plug the phone into the car's USB port. Gearslip opens by itself when the head unit connects.", "Plug into the car's USB port, or connect wirelessly. Gearslip opens by itself when the car answers.",
), ),
) )
@@ -23,6 +23,8 @@ class VideoSource(
private val height: Int, private val height: Int,
private val frameRate: Int, private val frameRate: Int,
private val mode: Mode = Mode.TEST_CARD, private val mode: Mode = Mode.TEST_CARD,
/** Over Wi-Fi: a fraction of the cable bitrate, so the link keeps up and sound isn't starved. */
private val wireless: Boolean = false,
private val onCodecConfig: (ByteArray) -> Unit, private val onCodecConfig: (ByteArray) -> Unit,
private val onFrame: (data: ByteArray, presentationTimeUs: Long, keyFrame: Boolean) -> Unit, private val onFrame: (data: ByteArray, presentationTimeUs: Long, keyFrame: Boolean) -> Unit,
) { ) {
@@ -45,7 +47,10 @@ class VideoSource(
private var frameIndex = 0L private var frameIndex = 0L
fun start() { fun start() {
val bitRate = (width.toLong() * height * frameRate * BITS_PER_PIXEL).toInt().coerceIn(MIN_BIT_RATE, MAX_BIT_RATE) val level = levelFor(width, height, frameRate)
val bitRate = defaultBitRate(width, height, frameRate, level).let {
if (wireless) (it * WIRELESS_SHARE).toInt().coerceAtMost(WIRELESS_MAX_BIT_RATE) else it
}
fun format(tuned: Boolean) = MediaFormat.createVideoFormat(MIME, width, height).apply { fun format(tuned: Boolean) = MediaFormat.createVideoFormat(MIME, width, height).apply {
setInteger( setInteger(
MediaFormat.KEY_COLOR_FORMAT, MediaFormat.KEY_COLOR_FORMAT,
@@ -60,8 +65,21 @@ class VideoSource(
// Drops already ask for a fresh keyframe, so a long interval costs nothing in recovery // Drops already ask for a fresh keyframe, so a long interval costs nothing in recovery
// and spares the link a big frame every second. // and spares the link a big frame every second.
setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, I_FRAME_INTERVAL_S) setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, I_FRAME_INTERVAL_S)
if (mode == Mode.SURFACE) {
// A surface encoder only makes a frame when the screen changes, and many decoders
// show a frame only once the next one arrives. So the last change before the
// screen goes still (a menu closing) sat in the car's decoder until something
// else moved. Repeating the last frame after a short pause pushes it through;
// a repeat of an unchanged screen costs a few bytes. scrcpy does the same.
setLong(MediaFormat.KEY_REPEAT_PREVIOUS_FRAME_AFTER, REPEAT_AFTER_US)
// An app that draws nonstop (a moving map) renders at the phone's refresh rate,
// faster than the car acks. Every extra frame queued inside the encoder, and the
// car's screen fell 25 seconds behind. Capping the input drops the extras before
// they're encoded, so the car always gets the newest frame.
setFloat(MediaFormat.KEY_MAX_FPS_TO_ENCODER, frameRate.toFloat())
}
setInteger(MediaFormat.KEY_PROFILE, MediaCodecInfo.CodecProfileLevel.AVCProfileBaseline) setInteger(MediaFormat.KEY_PROFILE, MediaCodecInfo.CodecProfileLevel.AVCProfileBaseline)
setInteger(MediaFormat.KEY_LEVEL, MediaCodecInfo.CodecProfileLevel.AVCLevel31) setInteger(MediaFormat.KEY_LEVEL, level.codecLevel)
if (tuned) { if (tuned) {
// Hints only: a car screen wants each frame now, not the best frame a moment later. // Hints only: a car screen wants each frame now, not the best frame a moment later.
setInteger(MediaFormat.KEY_BITRATE_MODE, MediaCodecInfo.EncoderCapabilities.BITRATE_MODE_VBR) setInteger(MediaFormat.KEY_BITRATE_MODE, MediaCodecInfo.EncoderCapabilities.BITRATE_MODE_VBR)
@@ -85,7 +103,7 @@ class VideoSource(
codec = encoder codec = encoder
running = true running = true
log.i("encoder started: ${width}x$height @ ${frameRate}fps, ${bitRate / 1000} kbps, H.264 Baseline, mode=$mode") log.i("encoder started: ${width}x$height @ ${frameRate}fps, ${bitRate / 1000} kbps${if (wireless) " (wireless)" else ""}, H.264 Baseline level ${level.label}, mode=$mode")
worker = Thread { worker = Thread {
runCatching { pump(encoder) } runCatching { pump(encoder) }
@@ -210,11 +228,50 @@ class VideoSource(
const val TIMEOUT_US = 10_000L const val TIMEOUT_US = 10_000L
const val SURFACE_TIMEOUT_US = 100_000L const val SURFACE_TIMEOUT_US = 100_000L
/** Enough for sharp text and map lines; 800x480 at 30fps comes out at 4 Mbps. */ /**
const val BITS_PER_PIXEL = 0.3 * 800x480 at 30fps gets 8 Mbps: about the most that size can show, enough that a moving
const val MIN_BIT_RATE = 4_000_000 * video in the browser stays sharp. Bigger screens get more, but less than in proportion,
const val MAX_BIT_RATE = 12_000_000 * since large frames compress better: bitrate grows with pixels^0.75. 60fps gets 1.5x,
* because frames that close together are mostly alike.
*/
const val BASE_BIT_RATE = 8_000_000.0
const val BASE_PIXELS = 800.0 * 480
/** No car's decoder is asked for more than this, whatever its level allows. */
const val MAX_BIT_RATE = 25_000_000
/**
* The H.264 levels a car screen can need, smallest first: macroblocks a second each
* decodes, and the Baseline bitrate it must handle. The encoder declares the smallest
* that fits, and the bitrate stays within it, so a car decoder isn't asked for more than
* the level it was told about.
*/
enum class Level(val label: String, val codecLevel: Int, val macroblocksPerSecond: Int, val maxBitRate: Int) {
L31("3.1", MediaCodecInfo.CodecProfileLevel.AVCLevel31, 108_000, 14_000_000),
L32("3.2", MediaCodecInfo.CodecProfileLevel.AVCLevel32, 216_000, 20_000_000),
L4("4", MediaCodecInfo.CodecProfileLevel.AVCLevel4, 245_760, 20_000_000),
L42("4.2", MediaCodecInfo.CodecProfileLevel.AVCLevel42, 522_240, 50_000_000),
}
fun levelFor(width: Int, height: Int, frameRate: Int): Level {
val perSecond = ((width + 15) / 16) * ((height + 15) / 16) * frameRate
return Level.entries.firstOrNull { perSecond <= it.macroblocksPerSecond } ?: Level.L42
}
fun defaultBitRate(width: Int, height: Int, frameRate: Int, level: Level): Int {
val size = Math.pow(width.toDouble() * height / BASE_PIXELS, 0.75)
val rate = if (frameRate > 30) 1.5 else 1.0
return (BASE_BIT_RATE * size * rate).toInt().coerceAtMost(minOf(level.maxBitRate, MAX_BIT_RATE))
}
/**
* A car's Wi-Fi is shared, noisy and far slower than its nominal rate, and the sound rides
* the same link behind the video. Real Android Auto sends a few Mbps wirelessly; at the
* cable rate (15-20 Mbps for 720p) a laptop hotspot fell seconds behind and the music
* cut in and out. 720p30 comes out near 5.4 Mbps.
*/
const val WIRELESS_SHARE = 0.35
const val WIRELESS_MAX_BIT_RATE = 8_000_000
const val I_FRAME_INTERVAL_S = 5 const val I_FRAME_INTERVAL_S = 5
private const val REPEAT_AFTER_US = 100_000L
/** (Y, Cb, Cr) - white, yellow, cyan, green, magenta, red, blue. */ /** (Y, Cb, Cr) - white, yellow, cyan, green, magenta, red, blue. */
val BAR_COLOURS = listOf( val BAR_COLOURS = listOf(
@@ -0,0 +1,94 @@
package app.seb3thehacker.gearslip.audio
import android.media.MediaCodec
import android.media.MediaCodecInfo
import android.media.MediaFormat
/**
* Turns captured PCM into AAC-LC for a car whose music channel takes nothing else. Cars that
* offer PCM never get here: AAC costs a little delay and quality, and every car that works today
* is a PCM car.
*
* Android's encoder hands back raw access units, which is what AUDIO_AAC_LC expects. For
* AUDIO_AAC_LC_ADTS each unit gets the 7-byte ADTS header in front, so the car can find the
* frame edges itself. The encoder's first output is its configuration, not audio, and is skipped.
*/
internal class AacEncoder(private val sampleRate: Int, private val channels: Int, private val adts: Boolean) {
private val codec: MediaCodec = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AAC).apply {
val format = MediaFormat.createAudioFormat(MediaFormat.MIMETYPE_AUDIO_AAC, sampleRate, channels).apply {
setInteger(MediaFormat.KEY_AAC_PROFILE, MediaCodecInfo.CodecProfileLevel.AACObjectLC)
setInteger(MediaFormat.KEY_BIT_RATE, if (channels == 1) 96_000 else 192_000)
setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 16 * 1024)
}
configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
start()
}
private val info = MediaCodec.BufferInfo()
private var samplesIn = 0L
/** Encodes [size] bytes of 16-bit PCM and returns the AAC frames that came out, oldest first. */
fun encode(pcm: ByteArray, size: Int): List<ByteArray> {
var offset = 0
val out = ArrayList<ByteArray>()
while (offset < size) {
val index = codec.dequeueInputBuffer(INPUT_WAIT_US)
if (index < 0) {
drain(out)
continue
}
val buffer = codec.getInputBuffer(index) ?: continue
val n = minOf(buffer.remaining(), size - offset)
buffer.put(pcm, offset, n)
val presentationUs = samplesIn * 1_000_000L / sampleRate
codec.queueInputBuffer(index, 0, n, presentationUs, 0)
samplesIn += n / (2 * channels)
offset += n
drain(out)
}
return out
}
private fun drain(out: MutableList<ByteArray>) {
while (true) {
val index = codec.dequeueOutputBuffer(info, 0)
if (index < 0) return
if (info.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG == 0 && info.size > 0) {
val buffer = codec.getOutputBuffer(index)
if (buffer != null) {
val frame = ByteArray(info.size)
buffer.position(info.offset)
buffer.get(frame)
out.add(if (adts) adtsHeader(frame.size) + frame else frame)
}
}
codec.releaseOutputBuffer(index, false)
}
}
/** ADTS for one AAC-LC frame: sync word, no CRC, and the frame's length including this header. */
private fun adtsHeader(payload: Int): ByteArray {
val length = payload + 7
val freq = FREQUENCIES.indexOf(sampleRate).let { if (it < 0) 3 else it }
val profile = 1 // AAC-LC is object type 2, written as type minus one
return byteArrayOf(
0xFF.toByte(),
0xF1.toByte(),
((profile shl 6) or (freq shl 2) or (channels shr 2)).toByte(),
(((channels and 3) shl 6) or (length shr 11)).toByte(),
((length shr 3) and 0xFF).toByte(),
(((length and 7) shl 5) or 0x1F).toByte(),
0xFC.toByte(),
)
}
fun release() {
runCatching { codec.stop() }
runCatching { codec.release() }
}
private companion object {
const val INPUT_WAIT_US = 10_000L
val FREQUENCIES = listOf(96000, 88200, 64000, 48000, 44100, 32000, 24000, 22050, 16000, 12000, 11025, 8000, 7350)
}
}
@@ -16,6 +16,7 @@ import android.media.AudioRecord
import android.media.projection.MediaProjection import android.media.projection.MediaProjection
import android.media.projection.MediaProjectionManager import android.media.projection.MediaProjectionManager
import android.os.IBinder import android.os.IBinder
import android.os.Process
import app.seb3thehacker.gearslip.GearslipLog import app.seb3thehacker.gearslip.GearslipLog
import kotlin.math.abs import kotlin.math.abs
@@ -147,6 +148,9 @@ class AudioCaptureService : Service() {
reading = true reading = true
thread = Thread({ thread = Thread({
// A busy phone (a map app drawing, the video encoder) would otherwise starve this
// loop, and the car hears every late read as a skip.
Process.setThreadPriority(Process.THREAD_PRIORITY_URGENT_AUDIO)
val buffer = ByteArray(CHUNK_BYTES) val buffer = ByteArray(CHUNK_BYTES)
var peak = 0 var peak = 0
var heardAt = System.currentTimeMillis() var heardAt = System.currentTimeMillis()
@@ -22,6 +22,15 @@ class AudioLink(
private val service: ServiceDiscovery.AudioService, private val service: ServiceDiscovery.AudioService,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit, private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
private val sendControl: (messageId: Int, body: ByteArray) -> Unit, private val sendControl: (messageId: Int, body: ByteArray) -> Unit,
/** Log name: "audio" for the media sink, "guidance" for the directions sink. */
private val tag: String = "audio",
private val sessionId: Int = SESSION_ID,
/** Media asks the car for focus; directions just start, and the car ducks music under them. */
private val asksFocus: Boolean = true,
private val onReady: (AudioLink) -> Unit = { CarAudio.attach(it) },
private val onClose: (AudioLink) -> Unit = { CarAudio.detach(it) },
/** The car took the sound back after giving it, as when the driver switches to the radio. */
private val onTakenBack: () -> Unit = {},
) : CarAudio.Link { ) : CarAudio.Link {
private var configIndex = 0 private var configIndex = 0
@@ -33,11 +42,15 @@ class AudioLink(
// The Uconnect asks for 4 on its video channel; assume the same until its Config says otherwise. // The Uconnect asks for 4 on its video channel; assume the same until its Config says otherwise.
@Volatile private var maxUnacked = 4 @Volatile private var maxUnacked = 4
@Volatile private var lastAckAt = 0L @Volatile private var lastAckAt = 0L
private var stalls = 0
private var chunks = 0 private var chunks = 0
private var bytes = 0L private var bytes = 0L
private var dropped = 0 private var dropped = 0
@Volatile private var configTimer: Thread? = null @Volatile private var configTimer: Thread? = null
private val ackLock = Object() private val ackLock = Object()
/** Only for a sink that takes AAC and nothing else; PCM sinks get the capture as it is. */
@Volatile private var aac: AacEncoder? = null
private val aacSink get() = service.codecType == ServiceDiscovery.CODEC_AAC_LC || service.codecType == ServiceDiscovery.CODEC_AAC_LC_ADTS
private val config get() = service.configs.getOrNull(configIndex) ?: service.configs.firstOrNull() private val config get() = service.configs.getOrNull(configIndex) ?: service.configs.firstOrNull()
override val sampleRate get() = config?.sampleRate ?: 48_000 override val sampleRate get() = config?.sampleRate ?: 48_000
@@ -46,28 +59,29 @@ class AudioLink(
val channelId get() = service.serviceId val channelId get() = service.serviceId
fun opened(status: Int) { fun opened(status: Int) {
GearslipLog.i("audio: <- ChannelOpenResponse (media audio): status=$status") GearslipLog.i("$tag: <- ChannelOpenResponse: status=$status")
if (status != 0) { if (status != 0) {
GearslipLog.w("audio: the head unit refused the media audio channel (status $status)") GearslipLog.w("$tag: the head unit refused the $tag channel (status $status)")
return return
} }
// AVChannelSetupRequest { required uint32 config_index = 1 }. This field is an index into // Setup { required MediaCodecType type = 1 }: the codec the car advertised for this sink
// the sink's own audio_configs, not a codec id - sending the codec type asks for a config // (aasdk's current protos, and LIVI reads it as the codec). Gearslip used to send 0 here,
// the sink does not have, and it answers with a status that never reaches READY. // read from an older proto as a config index; 0 is no codec at all, and the 2024 VW
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, 0L)) // answered UNEXPECTED_MESSAGE (id 255), never acked a chunk and played nothing.
GearslipLog.i("audio: -> Setup(config_index=0) on channel ${service.serviceId}, codec ${service.codecName}") sendOnChannel(MSG_SETUP, Protobuf.varintField(1, service.codecType.toLong()))
GearslipLog.i("$tag: -> Setup(codec=${service.codecName}) on channel ${service.serviceId}")
armConfigFallback() armConfigFallback()
} }
/** /**
* The 2018 Uconnect answers our Setup with a two-byte message we cannot identify (id 255, no * The 2018 Uconnect, a Dacia ULC 4.5 and the 2024 VW answered the old Setup (codec 0) with
* body) rather than the AVChannelSetupResponse the video channel gets, so the sink never * UNEXPECTED_MESSAGE (id 255, no body) rather than a Config, so the sink never reached READY.
* reaches READY and no audio is ever sent. * [MSG_SETUP_ANSWER] still takes that as the go-ahead, in case a car rejects the real codec too.
* *
* Service discovery already told us everything that message would have: the sink is PCM at * Service discovery already told us everything a Config would have: the sink's rate, channels
* 48000Hz stereo and it has exactly one configuration. So rather than wait forever for a * and its one configuration. So for a car that answers nothing at all, wait a moment and then
* message that is not coming, assume the advertised configuration and carry on. If that guess * assume the advertised configuration. If that guess is wrong the head unit simply will not
* is wrong the head unit simply will not ack, which the write path already reports. * ack, which the write path already reports.
*/ */
private fun armConfigFallback() { private fun armConfigFallback() {
val timer = Thread({ val timer = Thread({
@@ -76,19 +90,25 @@ class AudioLink(
} catch (e: InterruptedException) { } catch (e: InterruptedException) {
return@Thread return@Thread
} }
if (ready) return@Thread useAdvertisedSink("no usable Config after ${CONFIG_WAIT_MS}ms")
GearslipLog.w(
"audio: no usable Config after ${CONFIG_WAIT_MS}ms - falling back to the sink the " +
"head unit advertised (${sampleRate}Hz x$channels, config_index=0, max_unacked=$maxUnacked)",
)
configIndex = 0
ready = true
CarAudio.attach(this)
}, "gearslip-audio-config").apply { isDaemon = true } }, "gearslip-audio-config").apply { isDaemon = true }
configTimer = timer configTimer = timer
timer.start() timer.start()
} }
@Synchronized
private fun useAdvertisedSink(why: String) {
if (ready) return
configTimer?.interrupt()
GearslipLog.i(
"$tag: $why - using the sink the head unit advertised " +
"(${sampleRate}Hz x$channels, config_index=0, max_unacked=$maxUnacked)",
)
configIndex = 0
ready = true
onReady(this)
}
fun onMessage(messageId: Int, body: ByteArray) { fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) { when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> opened(Protobuf.readInt32Field(body, 1) ?: 0) MSG_CHANNEL_OPEN_RESPONSE -> opened(Protobuf.readInt32Field(body, 1) ?: 0)
@@ -98,17 +118,17 @@ class AudioLink(
val status = Wire.varint(fields, 1)?.toInt() val status = Wire.varint(fields, 1)?.toInt()
val window = Wire.varint(fields, 2)?.toInt() val window = Wire.varint(fields, 2)?.toInt()
val indices = fields.filter { it.number == 3 && it.wireType == 0 }.map { it.varint.toInt() } val indices = fields.filter { it.number == 3 && it.wireType == 0 }.map { it.varint.toInt() }
GearslipLog.i("audio: <- Config status=$status max_unacked=$window indices=$indices") GearslipLog.i("$tag: <- Config status=$status max_unacked=$window indices=$indices")
if (window != null) maxUnacked = maxOf(1, window) if (window != null) maxUnacked = maxOf(1, window)
configIndex = indices.firstOrNull() ?: 0 configIndex = indices.firstOrNull() ?: 0
if (status == STATUS_READY) { if (status == STATUS_READY) {
configTimer?.interrupt() configTimer?.interrupt()
ready = true ready = true
GearslipLog.i("audio: sink ready - ${sampleRate}Hz x$channels ${config?.bits}bit") GearslipLog.i("$tag: sink ready - ${sampleRate}Hz x$channels ${config?.bits}bit")
CarAudio.attach(this) onReady(this)
} else { } else {
GearslipLog.w( GearslipLog.w(
"audio: the sink is not ready (status=$status, 1=WAIT 2=READY) - " + "$tag: the sink is not ready (status=$status, 1=WAIT 2=READY) - " +
"nothing will be sent until it is", "nothing will be sent until it is",
) )
} }
@@ -123,9 +143,11 @@ class AudioLink(
} }
} }
// Whatever the head unit answers Setup with, we need its bytes to identify it. // Whatever else the head unit answers Setup with, we need its bytes to identify it.
else -> { else -> if (messageId == MSG_SETUP_ANSWER && body.isEmpty()) {
GearslipLog.i("audio: <- unhandled message id=$messageId (${body.size} bytes)") useAdvertisedSink("<- empty Setup answer (id $messageId)")
} else {
GearslipLog.i("$tag: <- unhandled message id=$messageId (${body.size} bytes)")
GearslipLog.hex(" audio msg", body, limit = 64) GearslipLog.hex(" audio msg", body, limit = 64)
} }
} }
@@ -138,17 +160,21 @@ class AudioLink(
*/ */
fun onFocus(state: Int) { fun onFocus(state: Int) {
val name = FOCUS_STATES[state] ?: "unknown($state)" val name = FOCUS_STATES[state] ?: "unknown($state)"
GearslipLog.i("audio: <- AudioFocusResponse state=$name") GearslipLog.i("$tag: <- AudioFocusResponse state=$name")
focusGranted = state == FOCUS_STATE_GAIN || state == FOCUS_STATE_GAIN_TRANSIENT val had = focusGranted
focusGranted = state == FOCUS_STATE_GAIN || state == FOCUS_STATE_GAIN_TRANSIENT ||
state == FOCUS_STATE_GAIN_MEDIA_ONLY
if (focusGranted) startStream() else stop() if (focusGranted) startStream() else stop()
// A ducking request leaves the music playing; a loss means another source has the speakers.
if (had && (state == FOCUS_STATE_LOSS || state == FOCUS_STATE_LOSS_TRANSIENT)) onTakenBack()
} }
override fun start() { override fun start() {
if (!ready || started) return if (!ready || started) return
if (!focusRequested) { if (asksFocus && !focusRequested) {
focusRequested = true focusRequested = true
sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_GAIN.toLong())) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_GAIN.toLong()))
GearslipLog.i("audio: -> AudioFocusRequest(GAIN); waiting for the head unit to hand the sink over") GearslipLog.i("$tag: -> AudioFocusRequest(GAIN); waiting for the head unit to hand the sink over")
} }
// Some head units grant the sink silently and never answer. Do not let that be the end of // Some head units grant the sink silently and never answer. Do not let that be the end of
// it: start anyway, and let the acks say whether anything is being taken. // it: start anyway, and let the acks say whether anything is being taken.
@@ -159,12 +185,18 @@ class AudioLink(
if (!ready || started) return if (!ready || started) return
sendOnChannel( sendOnChannel(
MSG_START, MSG_START,
Protobuf.varintField(1, SESSION_ID.toLong()) + Protobuf.varintField(2, configIndex.toLong()), Protobuf.varintField(1, sessionId.toLong()) + Protobuf.varintField(2, configIndex.toLong()),
) )
inFlight = 0 inFlight = 0
lastAckAt = System.currentTimeMillis() lastAckAt = System.currentTimeMillis()
if (aacSink && aac == null) {
aac = runCatching { AacEncoder(sampleRate, channels, adts = service.codecType == ServiceDiscovery.CODEC_AAC_LC_ADTS) }
.onSuccess { GearslipLog.i("$tag: the car takes ${service.codecName} only - encoding to AAC") }
.onFailure { GearslipLog.e("$tag: couldn't start the AAC encoder", it) }
.getOrNull()
}
started = true started = true
GearslipLog.i("audio: -> Start(session=$SESSION_ID, config=$configIndex) - streaming ${sampleRate}Hz x$channels") GearslipLog.i("$tag: -> Start(session=$sessionId, config=$configIndex) - streaming ${sampleRate}Hz x$channels")
} }
override fun stop() { override fun stop() {
@@ -172,18 +204,39 @@ class AudioLink(
started = false started = false
synchronized(ackLock) { ackLock.notifyAll() } synchronized(ackLock) { ackLock.notifyAll() }
runCatching { runCatching {
sendOnChannel(MSG_STOP, Protobuf.varintField(1, SESSION_ID.toLong())) sendOnChannel(MSG_STOP, Protobuf.varintField(1, sessionId.toLong()))
if (focusRequested) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_RELEASE.toLong())) if (focusRequested) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_RELEASE.toLong()))
} }
focusRequested = false focusRequested = false
focusGranted = false focusGranted = false
GearslipLog.i("audio: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks / $bytes bytes") aac?.release()
aac = null
GearslipLog.i("$tag: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks / $bytes bytes")
chunks = 0 chunks = 0
bytes = 0 bytes = 0
} }
override fun write(pcm: ByteArray, size: Int) { override fun write(pcm: ByteArray, size: Int) {
if (!started) return if (!started) return
val encoder = aac
if (encoder == null) {
send(pcm.copyOf(size))
return
}
val frames = try {
encoder.encode(pcm, size)
} catch (e: Exception) {
// A codec that dies mid-drive leaves the car silent, as it was before AAC; log it once.
GearslipLog.e("$tag: AAC encoding failed - no more audio this drive", e)
aac = null
started = false
return
}
frames.forEach(::send)
}
/** One Data message, once the car has room for it. */
private fun send(payload: ByteArray) {
// Wait for the car to catch up rather than throw the chunk away: a dropped chunk is a gap // Wait for the car to catch up rather than throw the chunk away: a dropped chunk is a gap
// the driver hears, while a short wait is soaked up by the capture buffer behind us. // the driver hears, while a short wait is soaked up by the capture buffer behind us.
synchronized(ackLock) { synchronized(ackLock) {
@@ -191,7 +244,11 @@ class AudioLink(
val waited = System.currentTimeMillis() - lastAckAt val waited = System.currentTimeMillis() - lastAckAt
if (waited > STALL_MS) { if (waited > STALL_MS) {
// A lost ack must not silence the stream for good; after a stall assume the window cleared. // A lost ack must not silence the stream for good; after a stall assume the window cleared.
GearslipLog.w("audio: no ack for ${STALL_MS}ms with $inFlight in flight - reopening the window") // A car that never acks (the 2024 VW, the Bolt) hits this every chunk: say so
// once, then now and then, so the rest of the log stays readable.
if (stalls++ % 500 == 0) {
GearslipLog.w("$tag: no ack for ${STALL_MS}ms with $inFlight in flight - reopening the window ($stalls times)")
}
inFlight = 0 inFlight = 0
break break
} }
@@ -206,20 +263,20 @@ class AudioLink(
val micros = System.nanoTime() / 1_000 val micros = System.nanoTime() / 1_000
for (i in 0 until 8) timestamp[i] = ((micros shr ((7 - i) * 8)) and 0xFF).toByte() for (i in 0 until 8) timestamp[i] = ((micros shr ((7 - i) * 8)) and 0xFF).toByte()
try { try {
sendOnChannel(MSG_DATA, timestamp + pcm.copyOf(size)) sendOnChannel(MSG_DATA, timestamp + payload)
} catch (e: java.io.IOException) { } catch (e: java.io.IOException) {
GearslipLog.e("audio: the link died mid-stream", e) GearslipLog.e("$tag: the link died mid-stream", e)
started = false started = false
return return
} }
synchronized(ackLock) { inFlight++ } synchronized(ackLock) { inFlight++ }
chunks++ chunks++
bytes += size bytes += payload.size
// The first chunk proves the whole path; after that, an occasional line is enough to show // The first chunk proves the whole path; after that, an occasional line is enough to show
// the car is still taking audio and to catch a window that has quietly closed. // the car is still taking audio and to catch a window that has quietly closed.
if (chunks == 1 || chunks % CHUNK_LOG_EVERY == 0) { if (chunks == 1 || chunks % CHUNK_LOG_EVERY == 0) {
GearslipLog.i( GearslipLog.i(
"audio: -> $chunks chunks / $bytes bytes, inFlight=$inFlight/$maxUnacked, " + "$tag: -> $chunks chunks / $bytes bytes, inFlight=$inFlight/$maxUnacked, " +
"dropped=$dropped, focus=${if (focusGranted) "granted" else "not granted"}", "dropped=$dropped, focus=${if (focusGranted) "granted" else "not granted"}",
) )
} }
@@ -230,7 +287,7 @@ class AudioLink(
configTimer = null configTimer = null
stop() stop()
ready = false ready = false
CarAudio.detach(this) onClose(this)
} }
companion object { companion object {
@@ -243,18 +300,24 @@ class AudioLink(
private const val MSG_STOP = 32770 private const val MSG_STOP = 32770
private const val MSG_CONFIG = 32771 private const val MSG_CONFIG = 32771
private const val MSG_ACK = 32772 private const val MSG_ACK = 32772
private const val MSG_SETUP_ANSWER = 255
private const val STATUS_READY = 2 private const val STATUS_READY = 2
private const val FOCUS_GAIN = 1 private const val FOCUS_GAIN = 1
private const val FOCUS_RELEASE = 4 private const val FOCUS_RELEASE = 4
private const val FOCUS_STATE_GAIN = 1 private const val FOCUS_STATE_GAIN = 1
private const val FOCUS_STATE_GAIN_TRANSIENT = 2 private const val FOCUS_STATE_GAIN_TRANSIENT = 2
private const val FOCUS_STATE_GAIN_MEDIA_ONLY = 6
private const val FOCUS_STATE_LOSS = 3
private const val FOCUS_STATE_LOSS_TRANSIENT = 5
private const val SESSION_ID = 2 private const val SESSION_ID = 2
private const val STALL_MS = 2_000L private const val STALL_MS = 2_000L
private const val CHUNK_LOG_EVERY = 200 // about every four seconds private const val CHUNK_LOG_EVERY = 200 // about every four seconds
private const val CONFIG_WAIT_MS = 2_500L private const val CONFIG_WAIT_MS = 1_000L
// AudioFocusStateNotification.AudioFocusState in aasdk.
private val FOCUS_STATES = mapOf( private val FOCUS_STATES = mapOf(
1 to "GAIN", 2 to "GAIN_TRANSIENT", 3 to "LOSS", 4 to "LOSS_TRANSIENT", 1 to "GAIN", 2 to "GAIN_TRANSIENT", 3 to "LOSS", 4 to "LOSS_TRANSIENT_CAN_DUCK",
5 to "LOSS_TRANSIENT", 6 to "GAIN_MEDIA_ONLY", 7 to "GAIN_TRANSIENT_GUIDANCE_ONLY",
) )
} }
} }
@@ -100,6 +100,26 @@ object CarAudio {
_capture.value = Capture.DENIED _capture.value = Capture.DENIED
} }
private val _micBlocked = MutableStateFlow(false)
/** The microphone permission is off for good: Android refuses without asking, so say why. */
val micBlocked: StateFlow<Boolean> = _micBlocked.asStateFlow()
fun micBlocked() {
_micBlocked.value = true
denied()
}
/** The driver turned the permission back on mid-drive: ask for capture again. */
fun micAllowed() {
if (!_micBlocked.value) return
_micBlocked.value = false
if (_capture.value == Capture.DENIED) {
_capture.value = Capture.OFF
if (wanted) request()
}
}
internal fun captureStarted() { internal fun captureStarted() {
_capture.value = Capture.ON _capture.value = Capture.ON
link?.start() link?.start()
@@ -0,0 +1,136 @@
package app.seb3thehacker.gearslip.audio
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
import app.seb3thehacker.gearslip.ServiceDiscovery
import app.seb3thehacker.gearslip.Wire
import java.util.concurrent.LinkedBlockingQueue
import java.util.concurrent.TimeUnit
/**
* The car's own microphone, which most head units offer over the same link as the screen:
*
* -> CHANNEL_OPEN_REQUEST <- CHANNEL_OPEN_RESPONSE
* -> Setup { config_index } <- Config
* -> VoiceSessionNotification { START } (control channel)
* -> MicrophoneRequest { open } <- MicrophoneResponse { status, session_id }
* <- Data { 8-byte timestamp, PCM }... -> Ack { session_id, ack }
* -> MicrophoneRequest { open = false }
* -> VoiceSessionNotification { END }
*
* The channel opens with the session, but the car only records between [start] and [stop], and
* only inside a voice session: the 2024 VW answered a bare MicrophoneRequest with status -7.
* Speech wants 16 kHz mono; a car that records at a multiple of that is averaged down to it, and
* one that doesn't isn't used, so voice falls back to the phone's microphone.
*/
class CarMic(
private val service: ServiceDiscovery.AudioService,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
private val voiceSession: (start: Boolean) -> Unit,
) {
val channelId get() = service.serviceId
private val config = service.configs.first()
private val step = config.sampleRate / RATE
private val usable = service.codecType == ServiceDiscovery.CODEC_PCM && config.bits == 16 &&
config.sampleRate % RATE == 0 && step >= 1 && config.channels in 1..2
@Volatile private var ready = false
@Volatile private var listening = false
@Volatile private var session = 0
private var packets = 0
private val chunks = LinkedBlockingQueue<ShortArray>()
fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
GearslipLog.i("mic: <- ChannelOpenResponse: status=$status")
if (status != 0) return
// Setup { type = the codec }, as for audio: 0 got UNEXPECTED_MESSAGE from the VW.
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, service.codecType.toLong()))
ready = usable
if (!usable) {
GearslipLog.w(
"mic: the car records ${service.codecName} ${config.sampleRate}Hz/${config.bits}bit/x${config.channels}, " +
"which speech can't use - voice stays on the phone's microphone",
)
}
}
MSG_CONFIG -> GearslipLog.i("mic: <- Config status=${Wire.varint(Wire.fields(body), 1)}")
MSG_MIC_RESPONSE -> {
val fields = Wire.fields(body)
session = Wire.varint(fields, 2)?.toInt() ?: 0
GearslipLog.i("mic: <- MicrophoneResponse status=${Wire.varint(fields, 1)} session=$session")
}
MSG_DATA -> {
sendOnChannel(MSG_ACK, Protobuf.varintField(1, session.toLong()) + Protobuf.varintField(2, 1L))
if (!listening || body.size <= TIMESTAMP_BYTES) return
if (packets++ == 0) GearslipLog.i("mic: first audio from the car (${body.size - TIMESTAMP_BYTES} bytes)")
chunks.offer(toSpeechRate(body))
}
else -> GearslipLog.i("mic: <- unhandled message id=$messageId (${body.size} bytes)")
}
}
/** Little-endian PCM after the timestamp, mixed to mono and averaged down to [RATE]. */
private fun toSpeechRate(body: ByteArray): ShortArray {
val frameBytes = 2 * config.channels
val frames = (body.size - TIMESTAMP_BYTES) / frameBytes
val out = ShortArray(frames / step)
for (i in out.indices) {
var sum = 0
for (s in 0 until step) {
val at = TIMESTAMP_BYTES + (i * step + s) * frameBytes
for (c in 0 until config.channels) {
val o = at + 2 * c
sum += (body[o].toInt() and 0xFF) or (body[o + 1].toInt() shl 8)
}
}
out[i] = (sum / (step * config.channels)).toShort()
}
return out
}
/** Asks the car to start recording. False when it can't, so the phone's microphone is used. */
fun start(): Boolean {
if (!ready) return false
chunks.clear()
packets = 0
listening = true
voiceSession(true)
sendOnChannel(MSG_MIC_REQUEST, Protobuf.varintField(1, 1L) + Protobuf.varintField(4, 1L))
GearslipLog.i("mic: -> MicrophoneRequest(open)")
return true
}
/** The next piece of audio, or null if the car sent nothing for [timeoutMs]. */
fun read(timeoutMs: Long): ShortArray? = chunks.poll(timeoutMs, TimeUnit.MILLISECONDS)
fun stop() {
if (!listening) return
listening = false
sendOnChannel(MSG_MIC_REQUEST, Protobuf.varintField(1, 0L))
voiceSession(false)
GearslipLog.i("mic: -> MicrophoneRequest(close) after $packets packets, voice session ended")
}
companion object {
/** The connected car's microphone, or null with no car or one that has none. */
@Volatile var current: CarMic? = null
const val RATE = 16_000
private const val TIMESTAMP_BYTES = 8
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_DATA = 0
private const val MSG_SETUP = 32768
private const val MSG_CONFIG = 32771
private const val MSG_ACK = 32772
private const val MSG_MIC_REQUEST = 32773
private const val MSG_MIC_RESPONSE = 32774
}
}
@@ -1,10 +1,14 @@
package app.seb3thehacker.gearslip.car package app.seb3thehacker.gearslip.car
import android.content.Context import android.content.Context
import app.seb3thehacker.gearslip.media.CarMedia
import app.seb3thehacker.gearslip.speech.VoiceFocus
import java.text.DateFormat import java.text.DateFormat
import java.util.Date import java.util.Date
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.delay
import kotlinx.coroutines.withContext import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeoutOrNull
/** /**
* Turns what [CarAssistant] heard into an action plus something to say back. * Turns what [CarAssistant] heard into an action plus something to say back.
@@ -17,9 +21,16 @@ import kotlinx.coroutines.withContext
object AssistantCommands { object AssistantCommands {
suspend fun handle(context: Context, navigator: CarNavigator, frame: CarEnvironment.Frame, heardRaw: String): String { suspend fun handle(context: Context, navigator: CarNavigator, frame: CarEnvironment.Frame, heardRaw: String): String {
val heard = heardRaw.lowercase().trim() val heard = natural(heardRaw)
VoiceHelp.hide()
if (heard in HELP_WORDS || heard.startsWith("what can i say") || heard.startsWith("what can you do")) {
VoiceHelp.show()
return "Here's what you can say."
}
if (heard.matches(Regex("(fuck|f\\*+) (you|off)( too)?"))) return "Fuck you too."
navigateCommand(context, navigator, frame, heard)?.let { return it } navigateCommand(context, navigator, frame, heard)?.let { return it }
playSearchCommand(context, navigator, heard)?.let { return it }
mediaCommand(heard)?.let { return it } mediaCommand(heard)?.let { return it }
navCommand(navigator, heard)?.let { return it } navCommand(navigator, heard)?.let { return it }
infoCommand(heard)?.let { return it } infoCommand(heard)?.let { return it }
@@ -29,6 +40,47 @@ object AssistantCommands {
return "I don't know how to do that yet." return "I don't know how to do that yet."
} }
/**
* What was said, minus the padding people wrap a command in: "hey", "can you", "I want to",
* "please", "thanks". Each command below then only has to know its own words.
*/
private fun natural(raw: String): String {
var t = raw.lowercase().replace(Regex("[.,!?]"), " ").replace(Regex("\\s+"), " ").trim()
var changed = true
while (changed) {
changed = false
for (lead in LEAD_FILLERS) if (t.startsWith("$lead ")) { t = t.removePrefix("$lead ").trim(); changed = true }
for (tail in TAIL_FILLERS) if (t.endsWith(" $tail")) { t = t.removeSuffix(" $tail").trim(); changed = true }
}
return t
}
/** The part after the first of [triggers] that [text] starts with, or null. */
private fun after(text: String, triggers: List<String>): String? =
triggers.firstOrNull { text.startsWith(it) }?.let { text.removePrefix(it).trim() }
private suspend fun allEntries(context: Context): List<Entry> {
val vehicleData = CarSettings.experimentalFeaturesEnabled.value && CarSettings.carSensorsEnabled.value
val builtIn = BuiltInApps.entries.filter { it.builtIn !== BuiltInApps.VehicleData || vehicleData }
return builtIn + (LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) })
}
/**
* Best label match: exact, then a label starting with [name], then one containing it. Spaces
* and punctuation don't count, since the recognizer may hear "MapQuest" as "map quest".
*/
private fun <T> bestMatch(items: List<T>, name: String, label: (T) -> String): T? {
val n = squash(name.removePrefix("the ").removePrefix("my ").removeSuffix(" app"))
if (n.isEmpty()) return null
items.firstOrNull { squash(label(it)) == n }?.let { return it }
// Loose matches need a real word: "pizza in a" mustn't pick "Organic Maps" for its "a".
if (n.length < 3) return null
return items.firstOrNull { squash(label(it)).startsWith(n) }
?: items.firstOrNull { squash(label(it)).contains(n) }
}
private fun squash(s: String) = s.lowercase().filter { it.isLetterOrDigit() }
/** /**
* "navigate to X [with|using|on|via Y]" - Y picks the nav app by (partial) label; without it, * "navigate to X [with|using|on|via Y]" - Y picks the nav app by (partial) label; without it,
* whichever nav app was used last, or the only one installed, or the first one found. Starts * whichever nav app was used last, or the only one installed, or the first one found. Starts
@@ -42,29 +94,49 @@ object AssistantCommands {
frame: CarEnvironment.Frame, frame: CarEnvironment.Frame,
heard: String, heard: String,
): String? { ): String? {
val trigger = NAVIGATE_TRIGGERS.firstOrNull { heard.startsWith(it) } ?: return null // "use Organic Maps to get to X", "open OsmAnd and navigate to X": the app comes first.
var destination = heard.removePrefix(trigger).trim() var text = heard
var leadingApp: String? = null
Regex("^(?:use|using|with|open|in|on) (.+?) (?:to |and )(.+)$").find(text)?.let { m ->
if (after(m.groupValues[2], NAVIGATE_TRIGGERS) != null) {
leadingApp = m.groupValues[1]
text = m.groupValues[2]
}
}
var destination = after(text, NAVIGATE_TRIGGERS) ?: return null
if (destination.isBlank()) return "Where to?" if (destination.isBlank()) return "Where to?"
var appName: String? = null val entries = allEntries(context)
for (sep in listOf(" with ", " using ", " on ", " via ")) { // "go to Spotify" means the app, and "go to settings" the screen; "go to Target" is a place.
val at = destination.lastIndexOf(sep) val asApp = destination.removePrefix("the ").removeSuffix(" app")
if (at > 0) { if (text.startsWith("go to ") &&
appName = destination.substring(at + sep.length).trim() (asApp in SCREEN_WORDS || asApp in APP_NICKNAMES || entries.any { it.label.lowercase() == asApp })
) {
return null
}
val navApps = entries.filter { it.template?.isNavigation == true }
if (navApps.isEmpty()) return "You don't have any navigation apps installed."
// "X with Organic Maps", "X on OsmAnd": split only when the end names a nav app, so a
// place like "Starbucks on Main Street" stays whole.
var chosenApp = leadingApp?.let { name -> bestMatch(navApps, name) { it.label } ?: return "I couldn't find a navigation app called $name." }
if (chosenApp == null) {
for (sep in listOf(" with ", " using ", " on ", " via ", " in ", " through ")) {
val at = destination.lastIndexOf(sep)
if (at <= 0) continue
val app = bestMatch(navApps, destination.substring(at + sep.length)) { it.label } ?: continue
chosenApp = app
destination = destination.substring(0, at).trim() destination = destination.substring(0, at).trim()
break break
} }
} }
// "the nearest gas station" -> "gas station": the map app already searches around the car
// and AutoRoute picks the nearest result, and Organic Maps took "nearest" as part of the name.
val (place, nearby) = plainPlace(destination)
destination = place
if (destination.isBlank()) return "Where to?" if (destination.isBlank()) return "Where to?"
val entries = LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) } val chosen = chosenApp ?: when {
val navApps = entries.filter { it.template?.isNavigation == true }
if (navApps.isEmpty()) return "You don't have any navigation apps installed."
val requestedApp = appName
val chosen = when {
requestedApp != null -> navApps.firstOrNull { it.label.lowercase().contains(requestedApp) }
?: return "I couldn't find a navigation app called $requestedApp."
navApps.size == 1 -> navApps.first() navApps.size == 1 -> navApps.first()
else -> navApps.firstOrNull { it.componentId == CarSettings.lastNav.value } ?: navApps.first() else -> navApps.firstOrNull { it.componentId == CarSettings.lastNav.value } ?: navApps.first()
} }
@@ -72,29 +144,104 @@ object AssistantCommands {
CarServices.connectNav(template, frame, destination) CarServices.connectNav(template, frame, destination)
navigator.home() navigator.home()
return "Navigating to $destination with ${chosen.label}." return if (nearby) "Finding the nearest $destination with ${chosen.label}." else "Navigating to $destination with ${chosen.label}."
}
/** [destination] without "a", "the nearest", "near me" and the like, and whether any of that said "close by". */
private fun plainPlace(destination: String): Pair<String, Boolean> {
var t = destination
var nearby = false
var changed = true
while (changed) {
changed = false
for (lead in PLACE_LEADS) if (t.startsWith(lead)) {
t = t.removePrefix(lead).trim(); changed = true
if (lead.trim() !in ARTICLES) nearby = true
}
for (tail in PLACE_TAILS) if (t.endsWith(tail)) { t = t.removeSuffix(tail).trim(); changed = true; nearby = true }
}
return t to nearby
}
/**
* "play X" with something after it: a song, artist, album or playlist, asked of the music app
* that's open through its own search ([CarMedia.search]). A bare "play", or "play music",
* falls through to resume.
*/
private suspend fun playSearchCommand(context: Context, navigator: CarNavigator, heard: String): String? {
var query = after(heard, PLAY_TRIGGERS) ?: return null
query = query.removePrefix("some ").trim()
if (query.isEmpty() || query in RESUME_WORDS) return null
val media = CarServices.media
// "play Daft Punk on Metrolist": switch to that app first, if it's one we know.
for (sep in listOf(" on ", " in ", " from ", " with ", " using ")) {
val at = query.lastIndexOf(sep)
if (at <= 0) continue
val apps = withContext(Dispatchers.IO) { app.seb3thehacker.gearslip.media.MediaCatalog.installed(context) }
val wanted = bestMatch(apps, query.substring(at + sep.length)) { it.label } ?: continue
query = query.substring(0, at).trim()
if (query.isEmpty() || query in RESUME_WORDS) {
CarServices.openMedia(wanted, autoplay = true)
navigator.media()
return "Opening ${wanted.label}."
}
if (CarServices.mediaApp.value?.component != wanted.component) {
CarServices.openMedia(wanted)
navigator.media()
// Give it a moment to connect before asking it to search.
withTimeoutOrNull(APP_CONNECT_MS) {
while (media.appLabel == null || (!media.canSearchLibrary && !media.now.value.canSearch)) delay(200)
}
}
break
}
// No music app open, or its connection is gone (the app was closed): use the one played
// last, the way "play X" works on Android Auto.
val phase = media.phase.value
if (media.appLabel == null || phase == CarMedia.Phase.IDLE || phase == CarMedia.Phase.REJECTED) {
if (!CarServices.reopenLastMedia()) return "Open a music app first, then ask again."
navigator.media()
}
withTimeoutOrNull(APP_CONNECT_MS) {
while (media.phase.value != CarMedia.Phase.READY || media.appLabel == null ||
(!media.canSearchLibrary && !media.now.value.canSearch)
) delay(200)
}
val app = media.appLabel ?: return "Your music app didn't open in time. Try again."
if (!media.canSearchLibrary && !media.now.value.canSearch) {
return "$app ignores voice searches. Pick something on its screen instead."
}
media.playBestMatch(query)
return "Looking for $query on $app."
} }
private fun mediaCommand(heard: String): String? { private fun mediaCommand(heard: String): String? {
val media = CarServices.media val media = CarServices.media
return when { return when {
"pause" in heard || heard.startsWith("stop") -> { "pause" in heard || heard.startsWith("stop") || heard == "quiet" || heard == "shut up" -> {
if (media.now.value.playing) media.togglePlay() media.pause()
VoiceFocus.afterRelease { media.pause() }
"Paused." "Paused."
} }
"resume" in heard || (heard.startsWith("play") && "playing" !in heard) -> { "resume" in heard || "unpause" in heard || heard in RESUME_PHRASES ||
(heard.startsWith("play") && "playing" !in heard) -> {
if (!media.now.value.playing) media.togglePlay() if (!media.now.value.playing) media.togglePlay()
"Playing." "Playing."
} }
"next" in heard || "skip" in heard -> { "next" in heard || "skip" in heard || heard.startsWith("change the song") -> {
media.next() media.next()
"Skipping." "Skipping."
} }
"previous" in heard || "last song" in heard || "back song" in heard -> { "previous" in heard || "last song" in heard || "back song" in heard || "go back a song" in heard ||
"play that again" in heard || "replay" in heard -> {
media.previous() media.previous()
"Playing the previous track." "Playing the previous track."
} }
"what" in heard && "playing" in heard -> { ("what" in heard && "playing" in heard) || "what song" in heard || "who sings" in heard ||
"who's singing" in heard || "who is singing" in heard || "what's this song" in heard ||
"name of this song" in heard -> {
val now = media.now.value val now = media.now.value
if (now.isActive) listOf(now.title, now.artist).filter { it.isNotBlank() }.joinToString(" by ") if (now.isActive) listOf(now.title, now.artist).filter { it.isNotBlank() }.joinToString(" by ")
else "Nothing's playing." else "Nothing's playing."
@@ -104,11 +251,19 @@ object AssistantCommands {
} }
private fun navCommand(navigator: CarNavigator, heard: String): String? = when { private fun navCommand(navigator: CarNavigator, heard: String): String? = when {
"go home" in heard || heard == "home" -> { heard == "go back" || heard == "back" -> {
navigator.back()
"Going back."
}
heard in EXIT_WORDS -> {
BuiltInApps.Exit.open(navigator)
"Back to the car's screen."
}
"go home" in heard || heard == "home" || heard == "home screen" || "go to the home screen" in heard -> {
navigator.home() navigator.home()
"Heading home." "Heading home."
} }
"open apps" in heard || heard == "apps" || "app list" in heard -> { "open apps" in heard || heard == "apps" || "app list" in heard || "all apps" in heard || "my apps" in heard || heard == "go to apps" -> {
navigator.apps() navigator.apps()
"Here are your apps." "Here are your apps."
} }
@@ -128,9 +283,10 @@ object AssistantCommands {
} }
private fun infoCommand(heard: String): String? = when { private fun infoCommand(heard: String): String? = when {
"what time" in heard || "what's the time" in heard -> "what time" in heard || "what's the time" in heard || heard == "time" || "the time" in heard ->
"It's ${DateFormat.getTimeInstance(DateFormat.SHORT).format(Date())}." "It's ${DateFormat.getTimeInstance(DateFormat.SHORT).format(Date())}."
"weather" in heard -> { "weather" in heard || "going to rain" in heard || "gonna rain" in heard || "how hot" in heard ||
"how cold" in heard || "temperature outside" in heard -> {
val data = (Weather.state.value as? WeatherState.Ready)?.data val data = (Weather.state.value as? WeatherState.Ready)?.data
if (data != null) "${data.temp}${data.tempUnit} and ${Weather.describe(data.code)} in ${data.place}." if (data != null) "${data.temp}${data.tempUnit} and ${Weather.describe(data.code)} in ${data.place}."
else "I don't have the weather yet." else "I don't have the weather yet."
@@ -139,12 +295,11 @@ object AssistantCommands {
} }
private suspend fun callCommand(context: Context, heard: String): String? { private suspend fun callCommand(context: Context, heard: String): String? {
val name = when { val name = (
heard.startsWith("call ") -> heard.removePrefix("call ") after(heard, CALL_TRIGGERS)
heard.startsWith("phone ") -> heard.removePrefix("phone ") ?: Regex("^give (.+) a (?:call|ring)$").find(heard)?.groupValues?.get(1)
heard.startsWith("dial ") -> heard.removePrefix("dial ") ?: return null
else -> return null ).replace(Regex(" (?:on|at) (?:their |his |her )?(?:mobile|cell|phone|home|work)$"), "").trim()
}.trim()
if (name.isBlank()) return "Who should I call?" if (name.isBlank()) return "Who should I call?"
if (!hasCallPermissions(context)) return "Gearslip needs the phone and contacts permissions first, from the phone screen." if (!hasCallPermissions(context)) return "Gearslip needs the phone and contacts permissions first, from the phone screen."
@@ -162,21 +317,69 @@ object AssistantCommands {
frame: CarEnvironment.Frame, frame: CarEnvironment.Frame,
heard: String, heard: String,
): String? { ): String? {
val name = when { val name = (after(heard, OPEN_TRIGGERS) ?: return null)
heard.startsWith("open ") -> heard.removePrefix("open ") .removePrefix("the ").removePrefix("my ").removeSuffix(" app").trim()
heard.startsWith("launch ") -> heard.removePrefix("launch ")
heard.startsWith("start ") -> heard.removePrefix("start ")
else -> return null
}.trim()
if (name.isBlank()) return null if (name.isBlank()) return null
val wanted = APP_NICKNAMES[name] ?: name
val entries = LauncherCache.entries ?: withContext(Dispatchers.IO) { LauncherCache.load(context) } // Gearslip's own apps come first in the list, so "phone" opens its dialer, not the phone's.
val match = entries.firstOrNull { it.label.lowercase().contains(name) } val match = bestMatch(allEntries(context), wanted) { it.label }
?: return "I couldn't find an app called $name." ?: return "I couldn't find an app called $name."
launchEntry(match, navigator, frame) launchEntry(match, navigator, frame)
return "Opening ${match.label}." return "Opening ${match.label}."
} }
private val NAVIGATE_TRIGGERS = listOf("navigate to ", "directions to ", "take me to ", "drive to ") private val EXIT_WORDS = setOf("exit", "exit gearslip", "close gearslip", "quit", "quit gearslip", "open exit")
/** Other names people use for Gearslip's own apps. */
private val APP_NICKNAMES = mapOf(
"web" to "browser", "internet" to "browser", "the web" to "browser", "web browser" to "browser",
"dialer" to "phone", "the dialer" to "phone", "keypad" to "phone",
"screen share" to "screen sharing", "screen mirroring" to "screen sharing", "mirroring" to "screen sharing",
)
private val NAVIGATE_TRIGGERS = listOf(
"navigate to ", "navigate me to ", "directions to ", "get directions to ", "give me directions to ",
"take me to ", "drive to ", "drive me to ", "get me to ", "go to ", "head to ", "route to ",
"how do i get to ", "show me the way to ", "find a route to ", "navigation to ", "get to ",
// Last, so "navigate to" wins: the recognizer sometimes drops the "to".
"navigate me ", "navigate ", "find me the nearest ", "find the nearest ", "find the closest ",
)
private val ARTICLES = setOf("a", "an", "the", "some")
// Longest first, so "the nearest " goes before "the ".
private val PLACE_LEADS = listOf(
"the nearest ", "the closest ", "a nearby ", "the nearby ", "nearest ", "closest ", "nearby ",
"a ", "an ", "some ",
)
private val PLACE_TAILS = listOf(" near me", " nearby", " near here", " around here", " close by", " closest to me")
private val PLAY_TRIGGERS = listOf("play me ", "play ", "put on ", "listen to ", "i want to hear ", "shuffle ")
private val CALL_TRIGGERS = listOf("call ", "phone ", "dial ", "ring ", "make a call to ", "place a call to ", "start a call with ")
private val OPEN_TRIGGERS = listOf("open ", "launch ", "start ", "go to ", "switch to ", "show me ", "bring up ", "pull up ")
/** Screens "go to X" means rather than a place. */
private val SCREEN_WORDS = setOf("settings", "home", "the home screen", "home screen", "apps", "my apps", "all apps")
/** Padding stripped from the front of a command, longest first where they overlap. */
private val LEAD_FILLERS = listOf(
"hey gearslip", "ok gearslip", "okay gearslip", "gearslip", "hey", "ok", "okay", "um", "uh", "so",
"please", "can you please", "can you", "could you", "would you", "will you", "can i", "could i",
"i want you to", "i want to", "i'd like to", "i would like to", "i wanna", "i need to", "i need you to",
"let's", "lets", "go ahead and", "just", "now", "quickly",
)
/** Longest first, so "right now" goes whole rather than leaving "right". */
private val TAIL_FILLERS = listOf("thank you very much", "thank you", "right now", "for me", "please", "thanks", "now")
private const val APP_CONNECT_MS = 5_000L
private val HELP_WORDS = setOf("help", "commands", "voice commands", "list commands", "show commands")
private val RESUME_WORDS = setOf("music", "the music", "some music", "something", "it", "again", "that", "my music")
private val RESUME_PHRASES = setOf("keep playing", "continue", "keep going", "start the music", "music on", "turn the music back on")
} }
@@ -4,7 +4,21 @@ import app.seb3thehacker.gearslip.car.theme.*
import android.content.Intent import android.content.Intent
import android.net.Uri import android.net.Uri
import android.provider.Settings import android.provider.Settings
import androidx.compose.animation.core.FastOutSlowInEasing
import androidx.compose.animation.core.RepeatMode
import androidx.compose.animation.core.animateFloat
import androidx.compose.animation.core.infiniteRepeatable
import androidx.compose.animation.core.rememberInfiniteTransition
import androidx.compose.animation.core.tween
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.ui.draw.drawBehind
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.scale
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.fillMaxWidth
@@ -30,47 +44,66 @@ import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
/** /**
* The band that appears while [CarAssistant] is listening, thinking or speaking - a thin strip * What the assistant hears and says, laid over the bottom of whatever screen is up: a dark shade
* across the top, same idea as [CallOverlay]'s active-call banner, so it never blocks the screen * so white words read over a bright map, and a glow along the bottom edge in the phone's own
* the driver was already looking at. * colours that breathes while listening. The shade passes touches through; only the buttons take them.
*/ */
@Composable @Composable
fun AssistantOverlay(modifier: Modifier = Modifier) { fun AssistantOverlay(modifier: Modifier = Modifier) {
val state by CarAssistant.state.collectAsState() val state by CarAssistant.state.collectAsState()
if (state.phase == CarAssistant.Phase.IDLE) return if (state.phase == CarAssistant.Phase.IDLE) return
val context = LocalContext.current val context = LocalContext.current
val listening = state.phase == CarAssistant.Phase.LISTENING && state.ready
val glow = MaterialTheme.colorScheme.primary
val glow2 = MaterialTheme.colorScheme.tertiary
val breathe by rememberInfiniteTransition(label = "glow").animateFloat(
0.55f, 1f, infiniteRepeatable(tween(900, easing = FastOutSlowInEasing), RepeatMode.Reverse), label = "glow",
)
val strength = when (state.phase) {
CarAssistant.Phase.LISTENING -> if (state.ready) breathe else 0.3f
CarAssistant.Phase.ERROR -> 0.3f
else -> 0.6f
}
Surface( Box(
color = MaterialTheme.colorScheme.primaryContainer, modifier
contentColor = MaterialTheme.colorScheme.onPrimaryContainer, .fillMaxWidth()
modifier = modifier.fillMaxWidth(), .drawBehind {
) { drawRect(Brush.verticalGradient(0f to Color.Transparent, 0.45f to Color.Black.copy(alpha = 0.6f), 1f to Color.Black.copy(alpha = 0.85f)))
Row( // A wide, flat oval of light rising from the bottom edge.
Modifier.padding(horizontal = 16.dp, vertical = 10.dp), scale(1f, 0.22f, Offset(size.width / 2, size.height)) {
verticalAlignment = Alignment.CenterVertically, drawCircle(
horizontalArrangement = Arrangement.spacedBy(12.dp), Brush.radialGradient(
) { 0f to glow.copy(alpha = 0.9f * strength), 0.5f to glow2.copy(alpha = 0.35f * strength), 1f to Color.Transparent,
PulsingMic(state.phase == CarAssistant.Phase.LISTENING) center = Offset(size.width / 2, size.height), radius = size.width * 0.55f,
Column(Modifier.weight(1f)) { ),
Text( radius = size.width * 0.55f, center = Offset(size.width / 2, size.height),
when (state.phase) {
CarAssistant.Phase.LISTENING -> "Listening…"
CarAssistant.Phase.THINKING -> "…"
CarAssistant.Phase.SPEAKING -> state.reply
CarAssistant.Phase.ERROR -> state.reply
CarAssistant.Phase.IDLE -> ""
},
style = MaterialTheme.typography.titleSmall,
fontWeight = FontWeight.SemiBold,
maxLines = 2,
)
if (state.heard.isNotBlank() && state.phase != CarAssistant.Phase.SPEAKING) {
Text(
"\"${state.heard}\"",
style = MaterialTheme.typography.bodySmall,
) )
} }
} }
.padding(start = 16.dp, end = 16.dp, top = 56.dp, bottom = 20.dp),
) {
Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(16.dp)) {
PulsingMic(listening)
Column(Modifier.weight(1f), horizontalAlignment = Alignment.CenterHorizontally) {
val (small, big) = when (state.phase) {
CarAssistant.Phase.LISTENING -> "" to state.heard.ifBlank { if (state.ready) "Listening…" else "One moment…" }
CarAssistant.Phase.THINKING -> "" to state.heard.ifBlank { "…" }
else -> state.heard to state.reply
}
if (small.isNotBlank()) {
Text(small, color = Color.White.copy(alpha = 0.7f), style = MaterialTheme.typography.bodyLarge, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
Text(
big,
color = if (big == "Listening…" || big == "One moment…" || big == "…") Color.White.copy(alpha = 0.7f) else Color.White,
style = MaterialTheme.typography.headlineSmall,
fontWeight = FontWeight.SemiBold,
textAlign = TextAlign.Center,
maxLines = 2,
overflow = TextOverflow.Ellipsis,
)
}
if (state.missingPermission) { if (state.missingPermission) {
GsButton(tone = GsTone.Tonal, onClick = { GsButton(tone = GsTone.Tonal, onClick = {
context.startActivity( context.startActivity(
@@ -0,0 +1,146 @@
package app.seb3thehacker.gearslip.car
import android.Manifest
import android.annotation.SuppressLint
import android.bluetooth.BluetoothA2dp
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothManager
import android.bluetooth.BluetoothProfile
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.PackageManager
import android.os.Handler
import android.os.Looper
import app.seb3thehacker.gearslip.GearslipLog
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* With Bluetooth audio chosen, music only reaches the car if the phone is connected to it over
* Bluetooth. Twenty seconds after the car shows Gearslip, long enough for a car to connect on its
* own, this checks. If the phone isn't connected, the car and the phone's Home screen both say so,
* until it connects or the drive ends.
*
* When the car named its Bluetooth address, only that device counts. Otherwise any audio
* connection does, since there's no telling which device is the car.
*/
object BluetoothAudioCheck {
private val _missing = MutableStateFlow(false)
/** True while Bluetooth audio is chosen and the phone isn't connected to the car for it. */
val missing: StateFlow<Boolean> = _missing
private val _carDismissed = MutableStateFlow(false)
/** The driver tapped OK on the car's card; it stays away for the rest of this drive. */
val carDismissed: StateFlow<Boolean> = _carDismissed
private val log = GearslipLog.tagged("bluetooth")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
private var armed = false
private var receiver: BroadcastReceiver? = null
fun init(context: Context) {
appContext = context.applicationContext
}
/** The car shows Gearslip. Called again whenever the video restarts, so it arms once per drive. */
fun onProjecting() = main.post {
if (armed) return@post
armed = true
main.postDelayed(::start, WAIT_MS)
}
fun onCarDisconnected() = main.post {
armed = false
main.removeCallbacksAndMessages(null)
stopWatching()
_missing.value = false
_carDismissed.value = false
}
fun dismissOnCar() {
_carDismissed.value = true
}
/** "Don't show again" on the car: turns the warning off in Settings, for every car. */
fun neverAgain() {
CarSettings.setBluetoothWarning(false)
log.i("Bluetooth audio warning turned off")
_carDismissed.value = true
_missing.value = false
}
private fun start() {
if (!armed) return
check()
// Connecting later clears the warning, and losing the connection brings it back.
val context = appContext ?: return
val r = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) {
main.postDelayed(::check, SETTLE_MS)
}
}
context.registerReceiver(
r,
IntentFilter(BluetoothA2dp.ACTION_CONNECTION_STATE_CHANGED).apply {
addAction(BluetoothAdapter.ACTION_STATE_CHANGED)
},
Context.RECEIVER_EXPORTED,
)
receiver = r
}
@SuppressLint("MissingPermission")
private fun check() {
if (!armed) return
val context = appContext ?: return
if (CarSettings.pipeAudio.value) {
set(false, "USB audio is on")
return
}
if (!CarSettings.bluetoothWarning.value) {
set(false, "the warning is turned off")
return
}
if (context.checkSelfPermission(Manifest.permission.BLUETOOTH_CONNECT) != PackageManager.PERMISSION_GRANTED) {
log.i("can't check the audio connection: the Bluetooth permission isn't granted")
return
}
val adapter = context.getSystemService(BluetoothManager::class.java)?.adapter ?: return
if (!adapter.isEnabled) {
set(true, "Bluetooth is off")
return
}
val car = CarBluetooth.carAddress?.uppercase()
adapter.getProfileProxy(context, object : BluetoothProfile.ServiceListener {
override fun onServiceConnected(profile: Int, proxy: BluetoothProfile) {
val connected = runCatching { proxy.connectedDevices }.getOrDefault(emptyList())
adapter.closeProfileProxy(profile, proxy)
val ok = if (car != null) connected.any { it.address.uppercase() == car } else connected.isNotEmpty()
main.post {
set(!ok, if (ok) "connected to the car" else "not connected to the car")
}
}
override fun onServiceDisconnected(profile: Int) {}
}, BluetoothProfile.A2DP)
}
private fun set(missing: Boolean, why: String) {
if (!armed || _missing.value == missing) return
_missing.value = missing
log.i("Bluetooth audio: $why")
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
private const val WAIT_MS = 20_000L
private const val SETTLE_MS = 1_000L
}
@@ -0,0 +1,118 @@
package app.seb3thehacker.gearslip.car
import app.seb3thehacker.gearslip.car.theme.*
import androidx.compose.foundation.background
import androidx.compose.foundation.clickable
import androidx.compose.foundation.interaction.MutableInteractionSource
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
/**
* Asks once per car whether to pair it over Bluetooth, then waits while the phone shows its
* pairing dialog. See [CarBluetooth]. With no pairing to do, says when Bluetooth audio isn't
* connected (see [BluetoothAudioCheck]), or else when the phone's location is off (see
* [LocationCheck]). Unlike the call prompt, a tap outside the card does nothing:
* the driver picks Pair or Not now.
*/
@Composable
internal fun BluetoothPairOverlay() {
val prompt by CarBluetooth.prompt.collectAsState()
val bluetoothAudio = !CarSettings.pipeAudio.collectAsState().value
when (prompt) {
CarBluetooth.Prompt.NONE -> {
val missing by BluetoothAudioCheck.missing.collectAsState()
val dismissed by BluetoothAudioCheck.carDismissed.collectAsState()
val warn by CarSettings.bluetoothWarning.collectAsState()
val locationOff by LocationCheck.off.collectAsState()
val locationDismissed by LocationCheck.carDismissed.collectAsState()
if (missing && bluetoothAudio && warn && !dismissed) PairCard(
title = "Bluetooth isn't connected",
body = "Music plays over Bluetooth, but your phone isn't connected to the car. " +
"Connect it from the car's Bluetooth menu or your phone's.",
secondary = "Don't show again" to BluetoothAudioCheck::neverAgain,
primary = "OK" to BluetoothAudioCheck::dismissOnCar,
) else if (locationOff && !locationDismissed) PairCard(
title = "Location is off",
body = "Map apps can't show where you are or give directions. Turn on location " +
"in your phone's quick settings when you're parked.",
secondary = "Don't show again" to LocationCheck::neverAgain,
primary = "OK" to LocationCheck::dismissOnCar,
)
}
CarBluetooth.Prompt.ASK -> PairCard(
title = "Pair with this car?",
body = (if (bluetoothAudio) "Calls and music play" else "Calls play") +
" through the car's speakers once your phone and the car are paired. Your phone " +
"then asks you to confirm a code.",
secondary = "Not now" to CarBluetooth::notNow,
primary = "Pair" to CarBluetooth::pair,
)
CarBluetooth.Prompt.PAIRING -> PairCard(
title = "Confirm on your phone",
body = "Check the code on your phone and tap Pair. The car may ask too.",
secondary = null,
primary = "Hide" to CarBluetooth::hide,
)
}
}
@Composable
private fun PairCard(title: String, body: String, secondary: Pair<String, () -> Unit>?, primary: Pair<String, () -> Unit>) {
Box(
Modifier.fillMaxSize().background(Color.Black.copy(alpha = 0.6f)).clickable(
indication = null,
interactionSource = remember { MutableInteractionSource() },
onClick = {},
),
contentAlignment = Alignment.Center,
) {
Surface(
shape = MaterialTheme.shapes.extraLarge,
color = MaterialTheme.colorScheme.surfaceContainerHigh,
modifier = Modifier.fillMaxWidth(0.85f).padding(16.dp),
) {
Column(
Modifier.padding(32.dp),
verticalArrangement = Arrangement.spacedBy(20.dp),
horizontalAlignment = Alignment.CenterHorizontally,
) {
Text(title, style = MaterialTheme.typography.headlineSmall, fontWeight = FontWeight.Bold)
Text(
body,
style = MaterialTheme.typography.titleMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
)
Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(16.dp)) {
secondary?.let { (label, onClick) ->
GsButton(onClick = onClick, tone = GsTone.Outline, modifier = Modifier.weight(1f).height(56.dp)) {
Text(label, style = MaterialTheme.typography.titleMedium)
}
}
GsButton(onClick = primary.second, modifier = Modifier.weight(1f).height(56.dp)) {
Text(primary.first, style = MaterialTheme.typography.titleMedium)
}
}
}
}
}
}
@@ -5,6 +5,11 @@ import android.view.SurfaceHolder
import android.view.SurfaceView import android.view.SurfaceView
import androidx.compose.foundation.gestures.detectTapGestures import androidx.compose.foundation.gestures.detectTapGestures
import androidx.compose.foundation.gestures.detectTransformGestures import androidx.compose.foundation.gestures.detectTransformGestures
import androidx.compose.foundation.gestures.awaitEachGesture
import androidx.compose.foundation.gestures.awaitFirstDown
import androidx.compose.ui.input.pointer.PointerEventPass
import androidx.compose.ui.input.pointer.util.VelocityTracker
import kotlin.math.hypot
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
@@ -125,6 +130,30 @@ private fun SurfaceHost(
if (pan != Offset.Zero) connection.surfaceScroll(-pan.x, -pan.y) if (pan != Offset.Zero) connection.surfaceScroll(-pan.x, -pan.y)
if (zoom != 1f) connection.surfaceScale(centroid.x, centroid.y, zoom) if (zoom != 1f) connection.surfaceScale(centroid.x, centroid.y, zoom)
} }
}
.pointerInput(connection) {
// detectTransformGestures has no velocity, so a fling is measured here alongside
// it: one finger lifting while still moving lets the map glide on, as on a car.
// Velocity is the finger's own, which is GestureDetector's onFling convention.
awaitEachGesture {
val down = awaitFirstDown(requireUnconsumed = false)
val tracker = VelocityTracker()
tracker.addPosition(down.uptimeMillis, down.position)
var pinched = false
while (true) {
val event = awaitPointerEvent(PointerEventPass.Final)
if (event.changes.size > 1) pinched = true
val change = event.changes.firstOrNull { it.id == down.id } ?: break
tracker.addPosition(change.uptimeMillis, change.position)
if (!change.pressed) {
val v = tracker.calculateVelocity()
if (!pinched && hypot(v.x, v.y) >= FLING_MIN_PX_S) {
connection.surfaceFling(v.x, v.y)
}
break
}
}
}
}, },
factory = { context -> factory = { context ->
SurfaceView(context).apply { SurfaceView(context).apply {
@@ -188,3 +217,6 @@ private fun ChromeBar(
/** Height of [ChromeBar]; the template's own chrome starts below it. */ /** Height of [ChromeBar]; the template's own chrome starts below it. */
private val CHROME_BAR = 44.dp private val CHROME_BAR = 44.dp
/** Slower than this when the finger lifts and it was a drag that stopped, not a fling. */
private const val FLING_MIN_PX_S = 400f
@@ -22,7 +22,7 @@ class CarApp(
object CarApps { object CarApps {
val all: List<CarApp> = listOf( val all: List<CarApp> = listOf(
CarApp("web", "Web", Icons.Filled.Search) { CarApp("web", "Browser", Icons.Filled.Search) {
WebApp(content = ProjectedContent.resolve(LocalContext.current)) WebApp(content = ProjectedContent.resolve(LocalContext.current))
}, },
CarApp("phone", "Screen sharing", Icons.Filled.Share) { CarApp("phone", "Screen sharing", Icons.Filled.Share) {
@@ -15,6 +15,7 @@ import androidx.core.content.ContextCompat
import app.seb3thehacker.gearslip.call.CallUiState import app.seb3thehacker.gearslip.call.CallUiState
import app.seb3thehacker.gearslip.call.CarCalls import app.seb3thehacker.gearslip.call.CarCalls
import app.seb3thehacker.gearslip.speech.SpeechEngine import app.seb3thehacker.gearslip.speech.SpeechEngine
import app.seb3thehacker.gearslip.speech.VoiceFocus
import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow import kotlinx.coroutines.flow.StateFlow
@@ -37,6 +38,8 @@ object CarAssistant {
val heard: String = "", val heard: String = "",
val reply: String = "", val reply: String = "",
val missingPermission: Boolean = false, val missingPermission: Boolean = false,
/** False from the press until the microphone is actually open, so the overlay doesn't claim to listen early. */
val ready: Boolean = false,
) )
private val _state = MutableStateFlow(State()) private val _state = MutableStateFlow(State())
@@ -88,6 +91,7 @@ object CarAssistant {
tts?.stop() tts?.stop()
_state.value = State(Phase.LISTENING) _state.value = State(Phase.LISTENING)
listenOnce( listenOnce(
onListening = { _state.value = _state.value.copy(ready = true) },
onPartial = { text -> _state.value = _state.value.copy(heard = text) }, onPartial = { text -> _state.value = _state.value.copy(heard = text) },
onHeard = { _state.value = _state.value.copy(phase = Phase.THINKING) }, onHeard = { _state.value = _state.value.copy(phase = Phase.THINKING) },
onResult = { text -> onResult = { text ->
@@ -119,15 +123,19 @@ object CarAssistant {
_state.value = _state.value.copy(phase = Phase.IDLE) _state.value = _state.value.copy(phase = Phase.IDLE)
return return
} }
appContext?.let(VoiceFocus::acquire)
engine.setOnUtteranceProgressListener(object : UtteranceProgressListener() { engine.setOnUtteranceProgressListener(object : UtteranceProgressListener() {
override fun onStart(utteranceId: String?) {} override fun onStart(utteranceId: String?) {}
override fun onDone(utteranceId: String?) { override fun onDone(utteranceId: String?) {
VoiceFocus.release()
_state.value = _state.value.copy(phase = Phase.IDLE) _state.value = _state.value.copy(phase = Phase.IDLE)
} }
@Deprecated("Deprecated in Java, no replacement on this API level") @Deprecated("Deprecated in Java, no replacement on this API level")
override fun onError(utteranceId: String?) { override fun onError(utteranceId: String?) {
VoiceFocus.release()
_state.value = _state.value.copy(phase = Phase.IDLE) _state.value = _state.value.copy(phase = Phase.IDLE)
} }
override fun onStop(utteranceId: String?, interrupted: Boolean) { VoiceFocus.release() }
}) })
engine.speak(text, TextToSpeech.QUEUE_FLUSH, null, "gearslip-assistant") engine.speak(text, TextToSpeech.QUEUE_FLUSH, null, "gearslip-assistant")
} }
@@ -138,17 +146,19 @@ object CarAssistant {
*/ */
fun say(text: String, onDone: () -> Unit) { fun say(text: String, onDone: () -> Unit) {
val engine = tts val engine = tts
if (engine == null || !ttsReady || text.isBlank()) { val context = appContext
if (engine == null || context == null || !ttsReady || text.isBlank()) {
mainHandler.post(onDone) mainHandler.post(onDone)
return return
} }
VoiceFocus.acquire(context)
engine.setOnUtteranceProgressListener(object : UtteranceProgressListener() { engine.setOnUtteranceProgressListener(object : UtteranceProgressListener() {
override fun onStart(utteranceId: String?) {} override fun onStart(utteranceId: String?) {}
override fun onDone(utteranceId: String?) { mainHandler.post(onDone) } override fun onDone(utteranceId: String?) { VoiceFocus.release(); mainHandler.post(onDone) }
@Deprecated("Deprecated in Java, no replacement on this API level") @Deprecated("Deprecated in Java, no replacement on this API level")
override fun onError(utteranceId: String?) { mainHandler.post(onDone) } override fun onError(utteranceId: String?) { VoiceFocus.release(); mainHandler.post(onDone) }
// Stopped part way: whoever stopped it has already moved on. // Stopped part way: whoever stopped it has already moved on.
override fun onStop(utteranceId: String?, interrupted: Boolean) {} override fun onStop(utteranceId: String?, interrupted: Boolean) { VoiceFocus.release() }
}) })
engine.speak(text, TextToSpeech.QUEUE_FLUSH, null, "gearslip-say") engine.speak(text, TextToSpeech.QUEUE_FLUSH, null, "gearslip-say")
} }
@@ -167,15 +177,16 @@ object CarAssistant {
onSpeaking: () -> Unit = {}, onSpeaking: () -> Unit = {},
) { ) {
cancel() cancel()
listenOnce(onPartial, onHeard = {}, onResult, onError, cue, onSpeaking) listenOnce(onPartial, onHeard = {}, onResult = onResult, onError = onError, cue = cue, onSpeaking = onSpeaking)
} }
/** /**
* One phrase, through Gearslip's own [SpeechEngine]. The phone's recognizer is only a fallback * One phrase, through Gearslip's own [SpeechEngine], or the phone's recognizer when the driver
* for when the engine itself can't run, since plenty of phones (GrapheneOS among them) have none. * chose it in settings. Never the phone's on its own: on most phones that's Google's.
*/ */
private fun listenOnce( private fun listenOnce(
onPartial: (String) -> Unit, onPartial: (String) -> Unit,
onListening: () -> Unit = {},
onHeard: () -> Unit, onHeard: () -> Unit,
onResult: (String) -> Unit, onResult: (String) -> Unit,
onError: (String) -> Unit, onError: (String) -> Unit,
@@ -183,24 +194,24 @@ object CarAssistant {
onSpeaking: () -> Unit = {}, onSpeaking: () -> Unit = {},
) { ) {
val context = appContext ?: return onError("Speech isn't ready yet.") val context = appContext ?: return onError("Speech isn't ready yet.")
if (CarSettings.systemSpeech.value && SpeechRecognizer.isRecognitionAvailable(context)) {
return systemListen(context, onListening, onPartial, onHeard, onResult, onError)
}
SpeechEngine.listen( SpeechEngine.listen(
context, context,
cue = cue, cue = cue,
onSpeaking = onSpeaking, onSpeaking = onSpeaking,
onPartial = onPartial,
onListening = onListening,
onHeard = onHeard, onHeard = onHeard,
onResult = onResult, onResult = onResult,
onError = { message -> onError = onError,
if (message == SpeechEngine.NOT_LOADED && SpeechRecognizer.isRecognitionAvailable(context)) {
systemListen(context, onPartial, onHeard, onResult, onError)
} else {
onError(message)
}
},
) )
} }
private fun systemListen( private fun systemListen(
context: Context, context: Context,
onListening: () -> Unit,
onPartial: (String) -> Unit, onPartial: (String) -> Unit,
onHeard: () -> Unit, onHeard: () -> Unit,
onResult: (String) -> Unit, onResult: (String) -> Unit,
@@ -209,7 +220,7 @@ object CarAssistant {
val r = SpeechRecognizer.createSpeechRecognizer(context) val r = SpeechRecognizer.createSpeechRecognizer(context)
recognizer = r recognizer = r
r.setRecognitionListener(object : RecognitionListener { r.setRecognitionListener(object : RecognitionListener {
override fun onReadyForSpeech(params: Bundle?) {} override fun onReadyForSpeech(params: Bundle?) = onListening()
override fun onBeginningOfSpeech() {} override fun onBeginningOfSpeech() {}
override fun onRmsChanged(rmsdB: Float) {} override fun onRmsChanged(rmsdB: Float) {}
override fun onBufferReceived(buffer: ByteArray?) {} override fun onBufferReceived(buffer: ByteArray?) {}
@@ -0,0 +1,255 @@
package app.seb3thehacker.gearslip.car
import android.Manifest
import android.annotation.SuppressLint
import android.bluetooth.BluetoothAdapter
import android.bluetooth.BluetoothDevice
import android.bluetooth.BluetoothManager
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.content.pm.PackageManager
import android.os.Handler
import android.os.Looper
import androidx.car.app.CarToast
import app.seb3thehacker.gearslip.AppSettings
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
import app.seb3thehacker.gearslip.ServiceDiscovery
import app.seb3thehacker.gearslip.Wire
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import java.security.MessageDigest
/**
* Pairs the phone with the car over Bluetooth the first time they meet, so calls (and music, when
* the driver chose Bluetooth audio) reach the car's speakers. Android Auto does this on its own;
* Gearslip asks first, on the car screen.
*
* The car names its Bluetooth address in its service list. When the driver taps Pair, Gearslip
* opens the car's Bluetooth channel and sends what Android Auto sends:
*
* -> CHANNEL_OPEN_REQUEST <- CHANNEL_OPEN_RESPONSE
* -> PairingRequest { phone_address, method }
* <- PairingResponse { status, already_paired }
*
* then starts pairing from the phone. Two things Android Auto has that Gearslip can't: the
* phone's own Bluetooth address (apps get a placeholder), and confirming the pairing code without
* asking. So a car may turn the request down, and the phone always shows its pairing dialog. The
* channel stays closed unless the driver taps Pair: every new channel is a risk to cars that work.
*/
object CarBluetooth {
enum class Prompt { NONE, ASK, PAIRING }
private val _prompt = MutableStateFlow(Prompt.NONE)
val prompt: StateFlow<Prompt> = _prompt
private val log = GearslipLog.tagged("bluetooth")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
/** This session's car: its channel, and how to open and write to it. */
private class Car(
val service: ServiceDiscovery.BluetoothService,
val open: () -> Unit,
val send: (messageId: Int, body: ByteArray) -> Unit,
)
@Volatile private var car: Car? = null
@Volatile private var bondStarted = false
/** From the Pair tap until it succeeds or fails, whether or not the card still shows. */
@Volatile private var pairing = false
private var receiver: BroadcastReceiver? = null
/** The car's Bluetooth address, when it named one in its service list. */
val carAddress: String? get() = car?.service?.carAddress
/** The car's Bluetooth channel once the driver has asked to pair, -1 otherwise. */
val channelId: Int get() = car?.takeIf { pairing }?.service?.serviceId ?: -1
fun init(context: Context) {
appContext = context.applicationContext
}
/** Called once the car lists its services. Asks on the car screen only if there's pairing to do. */
fun onCarConnected(
service: ServiceDiscovery.BluetoothService,
open: () -> Unit,
send: (messageId: Int, body: ByteArray) -> Unit,
) {
car = Car(service, open, send)
bondStarted = false
log.i("the car offers Bluetooth pairing, methods=${service.pairingMethods}")
val device = device() ?: return
when {
device.bondState == BluetoothDevice.BOND_BONDED -> log.i("already paired with this car")
declined(service.carAddress) -> log.i("not paired, and the driver said not now for this car")
else -> _prompt.value = Prompt.ASK
}
}
fun onCarDisconnected() {
car = null
pairing = false
main.post {
stopWatching()
_prompt.value = Prompt.NONE
}
}
/** The driver tapped Pair. */
fun pair() {
val c = car ?: return
val device = device() ?: return
pairing = true
_prompt.value = Prompt.PAIRING
watchBonding(device)
c.open()
log.i("-> ChannelOpenRequest on channel ${c.service.serviceId}")
// If the car never answers on its channel, pair from the phone anyway.
main.postDelayed({ startBond("the car didn't answer the pairing request") }, CAR_REPLY_WAIT_MS)
main.postDelayed({
if (pairing) finish("Pairing didn't finish. Pair from the car's Bluetooth menu instead.")
}, GIVE_UP_MS)
}
/** The driver tapped Not now: this car isn't asked about again. */
fun notNow() {
car?.service?.carAddress?.let { address ->
val context = appContext ?: return@let
val list = AppSettings.getString(context, KEY_DECLINED, "").split(",").filter { it.isNotEmpty() }
AppSettings.putString(context, KEY_DECLINED, (list + hash(address)).distinct().joinToString(","))
}
_prompt.value = Prompt.NONE
}
/** Hides the pairing card. The phone's own dialog carries on. */
fun hide() {
_prompt.value = Prompt.NONE
}
fun onMessage(messageId: Int, body: ByteArray) {
val c = car ?: return
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
log.i("<- ChannelOpenResponse: status=$status")
if (status != 0) {
main.post { startBond("the car wouldn't open its Bluetooth channel") }
return
}
val method = c.service.pairingMethods.firstOrNull { it == METHOD_NUMERIC_COMPARISON }
?: c.service.pairingMethods.firstOrNull() ?: METHOD_NUMERIC_COMPARISON
c.send(
MSG_PAIRING_REQUEST,
Protobuf.stringField(1, phoneAddress()) + Protobuf.varintField(2, method.toLong()),
)
log.i("-> PairingRequest(method=$method)")
}
MSG_PAIRING_RESPONSE -> {
val fields = Wire.fields(body)
log.i("<- PairingResponse status=${Wire.varint(fields, 1)} already_paired=${Wire.varint(fields, 2) == 1L}")
// Some cars start pairing themselves once they answer; give them a moment first.
main.postDelayed({ startBond("the car answered") }, CAR_BOND_WAIT_MS)
}
// Carries the pairing code. Android shows it to the driver, so it stays out of the log.
MSG_AUTHENTICATION_DATA -> log.i("<- AuthenticationData (${body.size} bytes)")
else -> log.i("<- unhandled message id=$messageId (${body.size} bytes)")
}
}
@SuppressLint("MissingPermission")
private fun startBond(why: String) {
if (bondStarted || !pairing) return
val device = device() ?: return
if (device.bondState != BluetoothDevice.BOND_NONE) return
bondStarted = true
val started = runCatching { device.createBond() }.getOrDefault(false)
log.i("pairing from the phone ($why): ${if (started) "started" else "refused"}")
if (!started) finish("Pairing didn't start. Pair from the car's Bluetooth menu instead.")
}
private fun watchBonding(device: BluetoothDevice) {
val context = appContext ?: return
stopWatching()
val r = object : BroadcastReceiver() {
@Suppress("DEPRECATION")
override fun onReceive(context: Context, intent: Intent) {
val changed = intent.getParcelableExtra<BluetoothDevice>(BluetoothDevice.EXTRA_DEVICE) ?: return
if (changed.address != device.address) return
val state = intent.getIntExtra(BluetoothDevice.EXTRA_BOND_STATE, BluetoothDevice.BOND_NONE)
val previous = intent.getIntExtra(BluetoothDevice.EXTRA_PREVIOUS_BOND_STATE, BluetoothDevice.BOND_NONE)
log.i("bond state $previous -> $state")
when {
state == BluetoothDevice.BOND_BONDING -> bondStarted = true
state == BluetoothDevice.BOND_BONDED -> finish("Paired. Calls now play through the car.")
previous == BluetoothDevice.BOND_BONDING -> finish("Pairing didn't finish. Pair from the car's Bluetooth menu instead.")
}
}
}
context.registerReceiver(r, IntentFilter(BluetoothDevice.ACTION_BOND_STATE_CHANGED), Context.RECEIVER_EXPORTED)
receiver = r
}
private fun finish(message: String) {
pairing = false
stopWatching()
main.removeCallbacksAndMessages(null)
_prompt.value = Prompt.NONE
CarToasts.show(this, message, CarToast.LENGTH_LONG)
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
/** The car as a Bluetooth device, or null (and logged) when Gearslip can't reach Bluetooth. */
private fun device(): BluetoothDevice? {
val context = appContext ?: return null
val address = car?.service?.carAddress ?: return null
if (context.checkSelfPermission(Manifest.permission.BLUETOOTH_CONNECT) != PackageManager.PERMISSION_GRANTED) {
log.i("can't check pairing: the Bluetooth permission isn't granted")
return null
}
val adapter = context.getSystemService(BluetoothManager::class.java)?.adapter
if (adapter == null || !adapter.isEnabled) {
log.i("can't check pairing: Bluetooth is off")
return null
}
if (!BluetoothAdapter.checkBluetoothAddress(address)) {
log.w("the car sent an address that isn't one")
return null
}
return adapter.getRemoteDevice(address)
}
/** Apps get 02:00:00:00:00:00 here, not the real address. Sent anyway, as the field is required. */
@SuppressLint("MissingPermission", "HardwareIds")
private fun phoneAddress(): String =
runCatching { appContext?.getSystemService(BluetoothManager::class.java)?.adapter?.address }.getOrNull()
?: "02:00:00:00:00:00"
private fun declined(address: String): Boolean {
val context = appContext ?: return false
return hash(address) in AppSettings.getString(context, KEY_DECLINED, "").split(",")
}
/** Settings keep a short hash, not the car's address. */
private fun hash(address: String): String =
MessageDigest.getInstance("SHA-256").digest(address.uppercase().toByteArray())
.take(8).joinToString("") { "%02x".format(it) }
private const val KEY_DECLINED = "bluetooth_pair_declined"
private const val CAR_REPLY_WAIT_MS = 5_000L
private const val CAR_BOND_WAIT_MS = 3_000L
private const val GIVE_UP_MS = 90_000L
private const val METHOD_NUMERIC_COMPARISON = 2
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_PAIRING_REQUEST = 32769
private const val MSG_PAIRING_RESPONSE = 32770
private const val MSG_AUTHENTICATION_DATA = 32771
}
@@ -60,7 +60,7 @@ fun CarDashboardScreen() {
.padding(horizontal = 24.dp, vertical = 16.dp), .padding(horizontal = 24.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(24.dp), verticalArrangement = Arrangement.spacedBy(24.dp),
) { ) {
Text("Today", style = MaterialTheme.typography.headlineMedium, fontWeight = FontWeight.Bold) ScreenHeader("Today")
WeatherCard(context) WeatherCard(context)
AgendaCard(context) AgendaCard(context)
NotificationsSection() NotificationsSection()
@@ -83,12 +83,13 @@ fun CarHome() {
val navigator = LocalCarNavigator.current val navigator = LocalCarNavigator.current
val panel = navigator.sidePanel val panel = navigator.sidePanel
// Coming back to the map from another screen: make sure the nav app is drawing into a live // Coming back to the map from another screen: the map stayed loaded underneath, so its surface
// surface. The first composition is skipped, since the surface is only just being lent. // is only replaced if it died meanwhile. The first composition is skipped, since the surface
// is only just being lent.
val onHome = navigator.current == CarScreen.Home val onHome = navigator.current == CarScreen.Home
val seenHome = remember { booleanArrayOf(false) } val seenHome = remember { booleanArrayOf(false) }
LaunchedEffect(onHome) { LaunchedEffect(onHome) {
if (onHome && seenHome[0]) CarServices.nav.refreshSurface() if (onHome && seenHome[0]) CarServices.nav.replaceDeadSurface()
if (onHome) seenHome[0] = true if (onHome) seenHome[0] = true
} }
val splitScreen = phase == CarMedia.Phase.READY && now.isActive && panel != SidePanel.NONE val splitScreen = phase == CarMedia.Phase.READY && now.isActive && panel != SidePanel.NONE
@@ -27,6 +27,13 @@ import androidx.compose.ui.graphics.vector.ImageVector
import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.runtime.compositionLocalOf
import androidx.compose.ui.platform.LocalConfiguration
import androidx.compose.ui.unit.Dp
import androidx.compose.ui.unit.coerceIn
import androidx.compose.runtime.rememberUpdatedState
import androidx.compose.runtime.LaunchedEffect
/** /**
* Whether a [CarKeyboard] is on screen right now, so [CarUi] can give it the row of height the * Whether a [CarKeyboard] is on screen right now, so [CarUi] can give it the row of height the
@@ -72,64 +79,77 @@ fun CarKeyboard(
onDispose { CarKeyboardVisibility.state.value = false } onDispose { CarKeyboardVisibility.state.value = false }
} }
val rows = if (symbols) SYMBOL_ROWS else LETTER_ROWS // The same field on the phone, for typing on its own keyboard: see [PhoneTyping].
val latestChange by rememberUpdatedState(onTextChange)
val latestSubmit by rememberUpdatedState(onSubmit)
val latestDismiss by rememberUpdatedState(onDismiss)
val phoneField = remember {
PhoneTyping.open(text, { latestChange(it) }, { latestSubmit() }, onDismiss?.let { { latestDismiss?.invoke() } })
}
DisposableEffect(phoneField) { onDispose { PhoneTyping.close(phoneField) } }
LaunchedEffect(text) { PhoneTyping.carText(phoneField, text) }
Surface( val rows = if (symbols) SYMBOL_ROWS else LETTER_ROWS
color = MaterialTheme.colorScheme.surfaceContainerHigh, val keyHeight = keyHeight(LocalConfiguration.current.screenHeightDp)
modifier = modifier.fillMaxWidth(),
) { CompositionLocalProvider(LocalKeyHeight provides keyHeight) {
Column( Surface(
Modifier.padding(horizontal = 6.dp, vertical = 4.dp), color = MaterialTheme.colorScheme.surfaceContainerHigh,
verticalArrangement = Arrangement.spacedBy(4.dp), modifier = modifier.fillMaxWidth(),
) { ) {
rows.dropLast(1).forEach { row -> Column(
Modifier.padding(horizontal = 6.dp, vertical = 4.dp),
verticalArrangement = Arrangement.spacedBy(4.dp),
) {
rows.dropLast(1).forEach { row ->
Row(
Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
row.forEach { key ->
val label = if (shifted && !symbols) key.uppercase() else key
Key(label, Modifier.weight(1f)) {
onTextChange(text + label)
shifted = false
}
}
}
}
// The bottom letter row: shift and backspace bookend it, exactly where a physical
// keyboard puts them relative to the letter block, not down beside the space bar.
Row( Row(
Modifier.fillMaxWidth(), Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(4.dp), horizontalArrangement = Arrangement.spacedBy(4.dp),
) { ) {
row.forEach { key -> if (!symbols) {
Key(if (shifted) "SHIFT" else "shift", Modifier.weight(1.6f)) { shifted = !shifted }
}
rows.last().forEach { key ->
val label = if (shifted && !symbols) key.uppercase() else key val label = if (shifted && !symbols) key.uppercase() else key
Key(label, Modifier.weight(1f)) { Key(label, Modifier.weight(1f)) {
onTextChange(text + label) onTextChange(text + label)
shifted = false shifted = false
} }
} }
} Key("⌫", Modifier.weight(1.6f)) {
} if (text.isNotEmpty()) onTextChange(text.dropLast(1))
// The bottom letter row: shift and backspace bookend it, exactly where a physical
// keyboard puts them relative to the letter block, not down beside the space bar.
Row(
Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(4.dp),
) {
if (!symbols) {
Key(if (shifted) "SHIFT" else "shift", Modifier.weight(1.6f)) { shifted = !shifted }
}
rows.last().forEach { key ->
val label = if (shifted && !symbols) key.uppercase() else key
Key(label, Modifier.weight(1f)) {
onTextChange(text + label)
shifted = false
} }
} }
Key("⌫", Modifier.weight(1.6f)) {
if (text.isNotEmpty()) onTextChange(text.dropLast(1))
}
}
// Control row: no backspace here - it stays with the letters above, not next to the // Control row: no backspace here - it stays with the letters above, not next to the
// key most likely to be hit right after it by mistake. Dismiss (when offered) is the // key most likely to be hit right after it by mistake. Search/enter takes the
// last key, bottom-right corner of the keyboard. // bottom-right corner, where a thumb expects it; dismiss (when offered) sits left of it.
Row( Row(
Modifier.fillMaxWidth(), Modifier.fillMaxWidth(),
horizontalArrangement = Arrangement.spacedBy(4.dp), horizontalArrangement = Arrangement.spacedBy(4.dp),
) { ) {
Key(if (symbols) "abc" else "?123", Modifier.weight(1.4f)) { symbols = !symbols } Key(if (symbols) "abc" else "?123", Modifier.weight(1.4f)) { symbols = !symbols }
Key("space", Modifier.weight(if (onDismiss != null) 3.2f else 4f)) { onTextChange("$text ") } Key("space", Modifier.weight(if (onDismiss != null) 3.2f else 4f)) { onTextChange("$text ") }
SubmitKey(Modifier.weight(1.6f), submitIcon, submitImage, onSubmit) if (onDismiss != null) {
if (onDismiss != null) { DismissKey(onDismiss, Modifier.weight(1.4f))
DismissKey(onDismiss, Modifier.weight(1.4f)) }
SubmitKey(Modifier.weight(1.6f), submitIcon, submitImage, onSubmit)
} }
} }
} }
@@ -140,7 +160,7 @@ fun CarKeyboard(
private fun Key(label: String, modifier: Modifier = Modifier, onClick: () -> Unit) { private fun Key(label: String, modifier: Modifier = Modifier, onClick: () -> Unit) {
GsIconBox( GsIconBox(
onClick = onClick, onClick = onClick,
modifier = modifier.height(KEY_HEIGHT), modifier = modifier.height(LocalKeyHeight.current),
colors = GsColors(MaterialTheme.colorScheme.surfaceContainerHighest, MaterialTheme.colorScheme.onSurface), colors = GsColors(MaterialTheme.colorScheme.surfaceContainerHighest, MaterialTheme.colorScheme.onSurface),
shape = RoundedCornerShape(8.dp), shape = RoundedCornerShape(8.dp),
) { ) {
@@ -163,7 +183,7 @@ private fun Key(label: String, modifier: Modifier = Modifier, onClick: () -> Uni
private fun DismissKey(onClick: () -> Unit, modifier: Modifier = Modifier) { private fun DismissKey(onClick: () -> Unit, modifier: Modifier = Modifier) {
GsIconBox( GsIconBox(
onClick = onClick, onClick = onClick,
modifier = modifier.height(KEY_HEIGHT), modifier = modifier.height(LocalKeyHeight.current),
colors = GsColors(MaterialTheme.colorScheme.surfaceContainerHighest, MaterialTheme.colorScheme.onSurfaceVariant), colors = GsColors(MaterialTheme.colorScheme.surfaceContainerHighest, MaterialTheme.colorScheme.onSurfaceVariant),
shape = RoundedCornerShape(8.dp), shape = RoundedCornerShape(8.dp),
) { ) {
@@ -181,7 +201,7 @@ private fun DismissKey(onClick: () -> Unit, modifier: Modifier = Modifier) {
private fun SubmitKey(modifier: Modifier, search: Boolean, image: ImageVector?, onClick: () -> Unit) { private fun SubmitKey(modifier: Modifier, search: Boolean, image: ImageVector?, onClick: () -> Unit) {
GsIconBox( GsIconBox(
onClick = onClick, onClick = onClick,
modifier = modifier.height(KEY_HEIGHT), modifier = modifier.height(LocalKeyHeight.current),
colors = GsColors(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.onPrimary), colors = GsColors(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.onPrimary),
shape = RoundedCornerShape(8.dp), shape = RoundedCornerShape(8.dp),
) { ) {
@@ -199,10 +219,12 @@ private fun SubmitKey(modifier: Modifier, search: Boolean, image: ImageVector?,
} }
/** /**
* Big enough to hit on a bumpy road, small enough that four rows plus a field plus a few * Big enough to hit on a bumpy road, small enough that four rows plus a field plus a few results
* results all fit a 480px-tall screen - which is the real constraint here, not the key size. * still fit. A 480-tall screen sets the floor; a taller one (LIVI's 1280x720) gets bigger keys.
*/ */
private val KEY_HEIGHT = 34.dp private fun keyHeight(screenHeightDp: Int): Dp = (screenHeightDp * 0.09f).dp.coerceIn(44.dp, 64.dp)
private val LocalKeyHeight = compositionLocalOf { 44.dp }
private val LETTER_ROWS = listOf( private val LETTER_ROWS = listOf(
"qwertyuiop".map { it.toString() }, "qwertyuiop".map { it.toString() },
@@ -27,6 +27,11 @@ object CarKeys {
private val _screenKeys = MutableSharedFlow<ScreenKey>(extraBufferCapacity = 8) private val _screenKeys = MutableSharedFlow<ScreenKey>(extraBufferCapacity = 8)
val screenKeys: SharedFlow<ScreenKey> = _screenKeys val screenKeys: SharedFlow<ScreenKey> = _screenKeys
private val _focusMoves = MutableSharedFlow<Unit>(extraBufferCapacity = 1)
/** A knob turn or D-pad press, so controls that hide when left alone can come back. */
val focusMoves: SharedFlow<Unit> = _focusMoves
/** Delivers an Android key event to the car display; set while the car UI is showing. */ /** Delivers an Android key event to the car display; set while the car UI is showing. */
@Volatile var focusSink: ((keyCode: Int, down: Boolean) -> Unit)? = null @Volatile var focusSink: ((keyCode: Int, down: Boolean) -> Unit)? = null
@@ -58,6 +63,7 @@ object CarKeys {
*/ */
fun onKey(keyCode: Int, down: Boolean, longPress: Boolean): Boolean { fun onKey(keyCode: Int, down: Boolean, longPress: Boolean): Boolean {
if (keyCode in FOCUS_KEYS) { if (keyCode in FOCUS_KEYS) {
_focusMoves.tryEmit(Unit)
focusSink?.invoke(keyCode, down) focusSink?.invoke(keyCode, down)
return true return true
} }
@@ -86,6 +92,7 @@ object CarKeys {
/** A turn of the rotary knob: each click moves focus one step, like Tab or Shift+Tab. */ /** A turn of the rotary knob: each click moves focus one step, like Tab or Shift+Tab. */
fun onRotate(delta: Int) { fun onRotate(delta: Int) {
val sink = focusSink ?: return val sink = focusSink ?: return
_focusMoves.tryEmit(Unit)
val code = if (delta > 0) KeyEvent.KEYCODE_TAB else KEYCODE_SHIFT_TAB val code = if (delta > 0) KeyEvent.KEYCODE_TAB else KEYCODE_SHIFT_TAB
repeat(kotlin.math.abs(delta).coerceAtMost(MAX_ROTARY_STEPS)) { repeat(kotlin.math.abs(delta).coerceAtMost(MAX_ROTARY_STEPS)) {
sink(code, true) sink(code, true)
@@ -95,15 +102,10 @@ object CarKeys {
/** /**
* The voice key: answers the message on screen if there is one, otherwise starts the * The voice key: answers the message on screen if there is one, otherwise starts the
* assistant when it's turned on. * assistant.
*/ */
private fun voice() = main.post { private fun voice() = main.post {
if (VoiceReply.onVoiceKey()) return@post if (!VoiceReply.onVoiceKey()) CarAssistant.start()
if (CarSettings.experimentalFeaturesEnabled.value && CarSettings.voiceAssistantEnabled.value) {
CarAssistant.start()
} else {
GearslipLog.i("keys: voice pressed, but the assistant is off in settings")
}
} }
/** Answers a ringing call; with nothing ringing, opens the phone screen. */ /** Answers a ringing call; with nothing ringing, opens the phone screen. */
@@ -16,6 +16,7 @@ import androidx.compose.foundation.lazy.grid.LazyVerticalGrid
import androidx.compose.foundation.lazy.grid.items import androidx.compose.foundation.lazy.grid.items
import androidx.compose.material.icons.Icons import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Info import androidx.compose.material.icons.filled.Info
import androidx.compose.material.icons.filled.KeyboardArrowDown
import androidx.compose.material3.Icon import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text import androidx.compose.material3.Text
@@ -47,6 +48,8 @@ import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.background import androidx.compose.foundation.background
import androidx.compose.foundation.layout.offset
import androidx.compose.foundation.border
import androidx.compose.foundation.shape.CircleShape import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.lazy.grid.GridItemSpan import androidx.compose.foundation.lazy.grid.GridItemSpan
import androidx.compose.material.icons.filled.Check import androidx.compose.material.icons.filled.Check
@@ -67,12 +70,8 @@ private class Tile(
val label: String, val label: String,
val icon: ImageBitmap? = null, val icon: ImageBitmap? = null,
val glyph: ImageVector? = null, val glyph: ImageVector? = null,
/** Tested from start to finish with Gearslip; drawn with a check mark. */
val verified: Boolean = false,
/** Known not to work; drawn with a red X instead of being left off the launcher. */ /** Known not to work; drawn with a red X instead of being left off the launcher. */
val broken: Boolean = false, val broken: Boolean = false,
/** Starts and can be used, but rough enough not to call working; drawn with a yellow mark. */
val partial: Boolean = false,
val onLongClick: (() -> Unit)? = null, val onLongClick: (() -> Unit)? = null,
val onClick: () -> Unit, val onClick: () -> Unit,
) )
@@ -133,10 +132,16 @@ internal object LauncherCache {
// own Messages screen does the same job, so a messenger gets only that tile. // own Messages screen does the same job, so a messenger gets only that tile.
val messagingPackages = messagingApps.map { it.packageName }.toSet() val messagingPackages = messagingApps.map { it.packageName }.toSet()
val nav = navApps.filter { it.component.packageName !in messagingPackages }.map { val nav = navApps
val label = if (it.component.packageName in mediaPackages) "${it.label} · Browse" else it.label .filter { it.component.packageName !in messagingPackages }
Entry(label, iconOf(it.component.packageName), template = it) // An app whose template screen is rejected but whose player works through Gearslip
} // (Spotify) shows only its player tile - a "· Browse" tile beside it would just report
// that the car screen isn't supported.
.filter { !(KnownApps.screenBroken(it.component.packageName) && KnownApps.playerWorks(it.component.packageName)) }
.map {
val label = if (it.component.packageName in mediaPackages) "${it.label} · Browse" else it.label
Entry(label, iconOf(it.component.packageName), template = it)
}
val media = mediaApps.map { Entry(it.label, iconOf(it.component.packageName), media = it) } val media = mediaApps.map { Entry(it.label, iconOf(it.component.packageName), media = it) }
// A messenger that is also a car media app keeps only its player tile. // A messenger that is also a car media app keeps only its player tile.
val messaging = messagingApps val messaging = messagingApps
@@ -157,6 +162,53 @@ internal fun rememberAllEntries(): List<Entry> {
return remember(installed) { BuiltInApps.entries + installed.orEmpty() } return remember(installed) { BuiltInApps.entries + installed.orEmpty() }
} }
/** The launcher's groups, in the order they're laid out. */
private enum class AppGroup { PHONE, MAPS, MUSIC, MESSAGING, WEB, SCREEN_SHARING, OTHER, SETTINGS, EXIT }
private fun groupOf(entry: Entry, mediaPackages: Set<String>): AppGroup {
entry.builtIn?.let {
return when (it) {
BuiltInApps.Phone -> AppGroup.PHONE
BuiltInApps.Web -> AppGroup.WEB
BuiltInApps.ScreenSharing -> AppGroup.SCREEN_SHARING
BuiltInApps.Exit -> AppGroup.EXIT
else -> AppGroup.SETTINGS
}
}
if (entry.media != null) return AppGroup.MUSIC
if (entry.messaging != null) return AppGroup.MESSAGING
val app = entry.template ?: return AppGroup.OTHER
// A media app's "· Browse" tile sits next to its player.
if (app.component.packageName in mediaPackages) return AppGroup.MUSIC
return when (app.kind) {
"calling" -> AppGroup.PHONE
"navigation", "poi", "parking", "charging" -> AppGroup.MAPS
// Template-only media apps may have no legacy browser/player tile to identify them.
"media" -> AppGroup.MUSIC
"messaging" -> AppGroup.MESSAGING
"settings" -> AppGroup.SETTINGS
else -> AppGroup.OTHER
}
}
/**
* The launcher's tiles in the order they're drawn: grouped by [AppGroup], alphabetical inside a
* group with Settings last of all, then with the driver's own moves from [custom] laid over it.
* Moved apps only trade places among themselves, so an app installed later still lands in its group.
*/
internal fun launcherOrder(all: List<Entry>, showVehicleData: Boolean, custom: List<String>): List<Entry> {
val shown = all.filter { it.builtIn !== BuiltInApps.VehicleData || showVehicleData }
val mediaPackages = shown.mapNotNull { it.media?.component?.packageName }.toSet()
val grouped = shown.sortedWith(
compareBy<Entry>({ groupOf(it, mediaPackages) }, { it.builtIn === BuiltInApps.Settings }, { it.label.lowercase() }),
)
if (custom.isEmpty()) return grouped
val rank = custom.withIndex().associate { (i, id) -> id to i }
fun Entry.rank() = componentId?.let { rank[it] }
val moved = grouped.filter { it.rank() != null }.sortedBy { it.rank() }.iterator()
return grouped.map { if (it.rank() != null) moved.next() else it }
}
/** An app by the id a pin or the running list knows it by, built-in or installed. */ /** An app by the id a pin or the running list knows it by, built-in or installed. */
internal fun findEntry(id: String): Entry? = internal fun findEntry(id: String): Entry? =
BuiltInApps.entries.find { it.componentId == id } ?: LauncherCache.entries?.find { it.componentId == id } BuiltInApps.entries.find { it.componentId == id } ?: LauncherCache.entries?.find { it.componentId == id }
@@ -184,9 +236,10 @@ internal fun launchEntry(entry: Entry, navigator: CarNavigator, frame: CarEnviro
entry.messaging?.let { navigator.messages(it.packageName, it.label) } entry.messaging?.let { navigator.messages(it.packageName, it.label) }
} }
/** The app launcher: Web and Screen sharing, every car app the phone has, then Settings, as an icon grid. */ /** The app launcher: every app as an icon grid, grouped Phone, Maps, Music, Messaging, Web, Screen sharing, Settings. */
@Composable @Composable
fun CarLauncher() { fun CarLauncher() {
val context = LocalContext.current
val navigator = LocalCarNavigator.current val navigator = LocalCarNavigator.current
val frame by CarEnvironment.frame.collectAsState() val frame by CarEnvironment.frame.collectAsState()
@@ -194,27 +247,21 @@ fun CarLauncher() {
val all = rememberAllEntries() val all = rememberAllEntries()
val running = rememberRunningApps() val running = rememberRunningApps()
val showVehicleData = rememberVehicleDataShown() val showVehicleData = rememberVehicleDataShown()
var showBadgeKey by remember { mutableStateOf(false) }
fun tileOf(entry: Entry): Tile { fun tileOf(entry: Entry): Tile {
// The badges rate an app's own car screens; a messenger shown on Gearslip's own Messages // Only apps that don't work get a mark. The X rates an app's own car screens; a messenger
// screen has none, so it gets no badge. // shown on Gearslip's own Messages screen has none, so it never gets one.
val pkg = entry.packageName?.takeIf { entry.messaging == null } val pkg = entry.packageName?.takeIf { entry.messaging == null }
val id = entry.componentId val id = entry.componentId
return Tile( return Tile(
entry.label, entry.icon, entry.builtIn?.glyph, entry.label, entry.icon, entry.builtIn?.glyph,
verified = pkg != null && (KnownApps.works(pkg) || (entry.media != null && KnownApps.playerWorks(pkg))), broken = pkg != null && (KnownApps.isBroken(context, pkg) || (entry.template != null && KnownApps.screenBroken(pkg))),
broken = pkg != null && (KnownApps.isBroken(pkg) || (entry.template != null && KnownApps.screenBroken(pkg))),
partial = pkg != null && KnownApps.isPartial(pkg),
onLongClick = id?.let { { navigator.showAppMenu(it) } }, onLongClick = id?.let { { navigator.showAppMenu(it) } },
) { openApp(entry, id?.let { running[it] }, navigator, frame) } ) { openApp(entry, id?.let { running[it] }, navigator, frame) }
} }
// Gearslip's own apps first, then everything installed, then Settings last where it's expected. val appOrder by CarSettings.appOrder.collectAsState()
val builtIns = all.filter { it.builtIn != null && (it.builtIn !== BuiltInApps.VehicleData || showVehicleData) } val tiles = launcherOrder(all, showVehicleData, appOrder).map(::tileOf)
val tiles = builtIns.filter { it.builtIn !== BuiltInApps.Settings }.map(::tileOf) +
all.filter { it.builtIn == null }.map(::tileOf) +
builtIns.filter { it.builtIn === BuiltInApps.Settings }.map(::tileOf)
LazyVerticalGrid( LazyVerticalGrid(
columns = GridCells.Adaptive(minSize = 108.dp), columns = GridCells.Adaptive(minSize = 108.dp),
@@ -224,60 +271,6 @@ fun CarLauncher() {
verticalArrangement = Arrangement.spacedBy(12.dp), verticalArrangement = Arrangement.spacedBy(12.dp),
) { ) {
items(tiles) { AppTile(it) } items(tiles) { AppTile(it) }
if (tiles.any { it.verified || it.broken || it.partial }) {
// Folded away behind one button: it's reference, read once, not something every
// visit to the launcher needs spelled out under the grid.
item(span = { GridItemSpan(maxLineSpan) }) {
Column(horizontalAlignment = Alignment.CenterHorizontally) {
GsButton(onClick = { showBadgeKey = !showBadgeKey }, tone = GsTone.Neutral) {
Icon(Icons.Filled.Info, contentDescription = null, modifier = Modifier.size(18.dp))
Spacer(Modifier.width(8.dp))
Text(if (showBadgeKey) "Hide badge key" else "What the badges mean")
}
if (showBadgeKey) {
BadgeLegend(
verified = tiles.any { it.verified },
partial = tiles.any { it.partial },
broken = tiles.any { it.broken },
)
}
}
}
}
}
}
/** Says what each badge on a tile means, once, under the grid - only the ones actually in use. */
@Composable
private fun BadgeLegend(verified: Boolean, partial: Boolean, broken: Boolean) {
Column(
Modifier.fillMaxWidth().padding(top = 4.dp),
horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.spacedBy(4.dp),
) {
if (verified) LegendRow(size = 16.dp, badge = { VerifiedBadge(it) }, text = "Tested and working with Gearslip")
if (partial) LegendRow(size = 16.dp, badge = { PartialBadge(it) }, text = "Runs, but not cleanly, with Gearslip")
if (broken) LegendRow(size = 16.dp, badge = { BrokenBadge(it) }, text = "Doesn't work with Gearslip yet")
}
}
@Composable
private fun LegendRow(size: androidx.compose.ui.unit.Dp, badge: @Composable (androidx.compose.ui.unit.Dp) -> Unit, text: String) {
Row(horizontalArrangement = Arrangement.Center, verticalAlignment = Alignment.CenterVertically) {
badge(size)
Spacer(Modifier.width(8.dp))
Text(text, style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
/** A green disc with a white check: the same mark on every tile and in the legend. */
@Composable
private fun VerifiedBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier = Modifier) {
Box(
modifier.size(size).background(Color(0xFF43A047), CircleShape),
contentAlignment = Alignment.Center,
) {
Icon(Icons.Filled.Check, contentDescription = "Tested and working", tint = Color.White, modifier = Modifier.size(size * 0.66f))
} }
} }
@@ -292,23 +285,12 @@ private fun BrokenBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier =
} }
} }
/** A yellow disc with a warning mark: neither the check nor the X, since this app runs but isn't clean about it. */
@Composable
private fun PartialBadge(size: androidx.compose.ui.unit.Dp, modifier: Modifier = Modifier) {
Box(
modifier.size(size).background(Color(0xFFF9A825), CircleShape),
contentAlignment = Alignment.Center,
) {
Icon(Icons.Filled.Warning, contentDescription = "Works, but not cleanly, with Gearslip", tint = Color.White, modifier = Modifier.size(size * 0.6f))
}
}
@OptIn(ExperimentalFoundationApi::class) @OptIn(ExperimentalFoundationApi::class)
@Composable @Composable
private fun AppTile(tile: Tile) { private fun AppTile(tile: Tile) {
val shape = MaterialTheme.shapes.extraLarge val shape = MaterialTheme.shapes.extraLarge
// A long press opens the app's menu (pin, close) - the same gesture everywhere apps are // A long press opens the app's menu (pin, close) - the same gesture everywhere apps are
// shown, here and on the nav bar. // shown, here and on the nav bar. GsIconBox marks it in the theme's style.
GsIconBox( GsIconBox(
onClick = tile.onClick, onClick = tile.onClick,
onLongClick = tile.onLongClick, onLongClick = tile.onLongClick,
@@ -322,29 +304,42 @@ private fun AppTile(tile: Tile) {
horizontalAlignment = Alignment.CenterHorizontally, horizontalAlignment = Alignment.CenterHorizontally,
verticalArrangement = Arrangement.Center, verticalArrangement = Arrangement.Center,
) { ) {
when { Box {
tile.icon != null -> Image(tile.icon, contentDescription = null, modifier = Modifier.size(52.dp)) TileIcon(tile)
tile.glyph != null -> Icon(tile.glyph, contentDescription = null, modifier = Modifier.size(52.dp)) if (tile.broken) {
// Pinned to the icon's lower corner, ringed in the tile's colour so it
// reads as sitting on top of the icon.
BrokenBadge(
22.dp,
Modifier.align(Alignment.BottomEnd).offset(x = 6.dp, y = 4.dp)
.border(2.dp, MaterialTheme.colorScheme.surfaceVariant, CircleShape),
)
}
} }
Spacer(Modifier.height(8.dp)) Spacer(Modifier.height(8.dp))
Text( TileLabel(tile)
tile.label,
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
textAlign = TextAlign.Center,
maxLines = 2,
overflow = TextOverflow.Ellipsis,
)
}
if (tile.verified) {
VerifiedBadge(24.dp, Modifier.align(Alignment.TopEnd).padding(8.dp))
}
if (tile.broken) {
BrokenBadge(24.dp, Modifier.align(Alignment.TopEnd).padding(8.dp))
}
if (tile.partial) {
PartialBadge(24.dp, Modifier.align(Alignment.TopEnd).padding(8.dp))
} }
} }
} }
} }
@Composable
private fun TileIcon(tile: Tile) {
when {
tile.icon != null -> Image(tile.icon, contentDescription = null, modifier = Modifier.size(52.dp))
tile.glyph != null -> Icon(tile.glyph, contentDescription = null, modifier = Modifier.size(52.dp))
}
}
@Composable
private fun TileLabel(tile: Tile, maxLines: Int = 2, modifier: Modifier = Modifier) {
Text(
tile.label,
modifier = modifier,
style = MaterialTheme.typography.labelLarge,
fontWeight = FontWeight.SemiBold,
textAlign = TextAlign.Center,
maxLines = maxLines,
overflow = TextOverflow.Ellipsis,
)
}
@@ -87,8 +87,8 @@ object SensorType {
18 to "Tire pressure", 18 to "Tire pressure",
19 to "Accelerometer", 19 to "Accelerometer",
20 to "Gyroscope", 20 to "Gyroscope",
21 to "Gravity", 21 to "GPS satellites",
22 to "Wheel tick / distance", 22 to "Toll card",
) )
fun name(id: Int): String = NAMES[id]?.let { "$it (#$id)" } ?: "Sensor #$id" fun name(id: Int): String = NAMES[id]?.let { "$it (#$id)" } ?: "Sensor #$id"
@@ -33,6 +33,12 @@ object CarServices {
val nav: CarAppConnection by lazy { CarAppConnection(requireNotNull(context)) } val nav: CarAppConnection by lazy { CarAppConnection(requireNotNull(context)) }
val media: CarMedia by lazy { CarMedia(requireNotNull(context)) } val media: CarMedia by lazy { CarMedia(requireNotNull(context)) }
/** [nav], or null before the car screen has set things up. */
fun navOrNull(): CarAppConnection? = if (context != null) nav else null
/** [media], or null before the car screen has set things up: for code outside the car UI. */
fun mediaOrNull(): CarMedia? = if (context != null) media else null
/** /**
* A second, independent connection for templated apps that aren't navigation apps - Spotify's * A second, independent connection for templated apps that aren't navigation apps - Spotify's
* own Car App Library service is the first of these. Kept apart from [nav] so opening one * own Car App Library service is the first of these. Kept apart from [nav] so opening one
@@ -59,6 +65,10 @@ object CarServices {
app.seb3thehacker.gearslip.notify.CarMotion.start(appContext) app.seb3thehacker.gearslip.notify.CarMotion.start(appContext)
app.seb3thehacker.gearslip.call.CarCalls.start(appContext) app.seb3thehacker.gearslip.call.CarCalls.start(appContext)
CarAssistant.init(appContext) CarAssistant.init(appContext)
DirectionsVoice.init(appContext)
CarBluetooth.init(appContext)
BluetoothAudioCheck.init(appContext)
LocationCheck.init(appContext)
CarKeys.init(appContext) CarKeys.init(appContext)
VoiceReply.init(appContext) VoiceReply.init(appContext)
// Loaded now, while nothing's waiting on it, so the first reply isn't slower than the rest. // Loaded now, while nothing's waiting on it, so the first reply isn't slower than the rest.
@@ -79,15 +89,22 @@ object CarServices {
if (destination != null && nav.connectedComponent == app.component) { if (destination != null && nav.connectedComponent == app.component) {
nav.navigateTo(destination) nav.navigateTo(destination)
} else { } else {
// Another map app taking over ends the old one's route, or it keeps guiding unseen.
if (nav.connectedComponent != app.component) endRoute()
nav.connect(app, frame.width, frame.height, frame.densityDpi, destination) nav.connect(app, frame.width, frame.height, frame.densityDpi, destination)
} }
} }
fun stopNav() { fun stopNav() {
navStopped = true navStopped = true
endRoute()
nav.disconnect() nav.disconnect()
} }
private fun endRoute() {
if (nav.navigating) nav.stopNavigating()
}
/** Undoes [stopNav]'s hold so the last map app comes back, same as it would after a restart. */ /** Undoes [stopNav]'s hold so the last map app comes back, same as it would after a restart. */
suspend fun reconnectNav(frame: CarEnvironment.Frame) { suspend fun reconnectNav(frame: CarEnvironment.Frame) {
navStopped = false navStopped = false
@@ -121,6 +138,19 @@ object CarServices {
if (CarSettings.pipeAudio.value) CarAudio.request() if (CarSettings.pipeAudio.value) CarAudio.request()
} }
/**
* Reopens the media app used last, for the car's media button when nothing is open. False
* when there's none on record or it's been uninstalled since.
*/
suspend fun reopenLastMedia(): Boolean {
val ctx = context ?: return false
val last = CarSettings.lastMedia.value?.let(ComponentName::unflattenFromString) ?: return false
val app = withContext(Dispatchers.IO) { MediaCatalog.installed(ctx) }
.firstOrNull { it.component == last } ?: return false
openMedia(app)
return true
}
/** Closes the media app from the car: stops what it's playing and drops the connection. */ /** Closes the media app from the car: stops what it's playing and drops the connection. */
fun closeMedia() { fun closeMedia() {
if (media.now.value.playing) media.togglePlay() if (media.now.value.playing) media.togglePlay()
@@ -148,7 +178,7 @@ object CarServices {
_hasNavApps.value = navApps.isNotEmpty() _hasNavApps.value = navApps.isNotEmpty()
// Never auto-pick one already known to fail outright - a driver who plugs in should // Never auto-pick one already known to fail outright - a driver who plugs in should
// land on a map, not on the one tile marked with a red X. // land on a map, not on the one tile marked with a red X.
val candidates = navApps.filterNot { KnownApps.isBroken(it.component.packageName) } val candidates = navApps.filterNot { KnownApps.isBroken(ctx, it.component.packageName) }
val last = CarSettings.lastNav.value val last = CarSettings.lastNav.value
// The one used last, or - first run, nothing on record yet - whichever comes first: // The one used last, or - first run, nothing on record yet - whichever comes first:
// a driver who plugs in for the first time shouldn't have to go pick one by hand // a driver who plugs in for the first time shouldn't have to go pick one by hand
@@ -194,6 +224,32 @@ object CarServices {
return playingPackages.firstNotNullOfOrNull { pkg -> catalog.firstOrNull { it.component.packageName == pkg } } return playingPackages.firstNotNullOfOrNull { pkg -> catalog.firstOrNull { it.component.packageName == pkg } }
} }
/**
* The car went away mid-song: pause it, as Android Auto does, so it doesn't carry on out of
* the phone's speaker. Unplugging a USB accessory isn't "audio becoming noisy" the way
* headphones are, so the apps won't pause themselves. Every playing session is paused, not
* just the one Gearslip opened: anything the car was playing was going through the car.
*/
fun pausePhoneMedia(why: String = "the car left") {
val ctx = context ?: return
val manager = ctx.getSystemService(MediaSessionManager::class.java)
val listener = ComponentName(ctx, GearslipNotificationListener::class.java)
val playing = try {
manager.getActiveSessions(listener).filter { it.playbackState?.state == PlaybackState.STATE_PLAYING }
} catch (e: SecurityException) {
null
}
if (playing == null) {
// No notification access: the app Gearslip itself is driving is all it can reach.
mediaOrNull()?.let { if (it.now.value.playing) it.togglePlay() }
return
}
playing.forEach { runCatching { it.transportControls.pause() } }
if (playing.isNotEmpty()) {
app.seb3thehacker.gearslip.GearslipLog.i("media: paused ${playing.joinToString { it.packageName }}: $why")
}
}
/** The car went away: nothing should keep running for a screen that no longer exists. */ /** The car went away: nothing should keep running for a screen that no longer exists. */
fun shutdown() { fun shutdown() {
context?.let { app.seb3thehacker.gearslip.notify.CarMotion.stop(it) } context?.let { app.seb3thehacker.gearslip.notify.CarMotion.stop(it) }
@@ -202,6 +258,8 @@ object CarServices {
runCatching { media.disconnect() } runCatching { media.disconnect() }
runCatching { browse.disconnect() } runCatching { browse.disconnect() }
CarAssistant.cancel() CarAssistant.cancel()
// About 900 MB that nothing needs once the car's gone.
app.seb3thehacker.gearslip.speech.SpeechEngine.unload()
CarAudio.release() CarAudio.release()
_mediaApp.value = null _mediaApp.value = null
navStopped = false navStopped = false
@@ -2,6 +2,7 @@ package app.seb3thehacker.gearslip.car
import android.content.Context import android.content.Context
import app.seb3thehacker.gearslip.AppSettings import app.seb3thehacker.gearslip.AppSettings
import app.seb3thehacker.gearslip.audio.CarAudio
import app.seb3thehacker.gearslip.car.theme.GsTheme import app.seb3thehacker.gearslip.car.theme.GsTheme
import app.seb3thehacker.gearslip.car.theme.GsThemes import app.seb3thehacker.gearslip.car.theme.GsThemes
import kotlinx.coroutines.flow.MutableStateFlow import kotlinx.coroutines.flow.MutableStateFlow
@@ -27,6 +28,10 @@ object CarSettings {
private const val KEY_OPEN = "car_open_on_connect" private const val KEY_OPEN = "car_open_on_connect"
private const val KEY_NIGHT = "car_night_mode" private const val KEY_NIGHT = "car_night_mode"
private const val KEY_PIPE_AUDIO = "car_pipe_audio" private const val KEY_PIPE_AUDIO = "car_pipe_audio"
private const val KEY_SPEAK_DIRECTIONS = "car_speak_directions"
private const val KEY_BLUETOOTH_WARNING = "car_bluetooth_warning"
private const val KEY_LOCATION_WARNING = "car_location_warning"
private const val KEY_AUDIO_CHOICE_ASKED = "car_audio_choice_asked"
private const val KEY_LAST_NAV = "car_last_nav" private const val KEY_LAST_NAV = "car_last_nav"
private const val KEY_LAST_MEDIA = "car_last_media" private const val KEY_LAST_MEDIA = "car_last_media"
private const val KEY_AUTOPLAY = "car_autoplay" private const val KEY_AUTOPLAY = "car_autoplay"
@@ -34,8 +39,9 @@ object CarSettings {
private const val KEY_THEME = "car_app_theme" private const val KEY_THEME = "car_app_theme"
private const val KEY_UI_THEME = "car_ui_theme" private const val KEY_UI_THEME = "car_ui_theme"
private const val KEY_PINNED = "car_pinned_apps" private const val KEY_PINNED = "car_pinned_apps"
private const val KEY_APP_ORDER = "car_app_order"
private const val KEY_EXPERIMENTAL_FEATURES = "car_experimental_features" private const val KEY_EXPERIMENTAL_FEATURES = "car_experimental_features"
private const val KEY_VOICE_ASSISTANT = "car_voice_assistant" private const val KEY_SYSTEM_SPEECH = "car_system_speech"
private const val KEY_CAR_SENSORS = "car_sensors_enabled" private const val KEY_CAR_SENSORS = "car_sensors_enabled"
private const val KEY_SEEN_SAFETY_WARNING = "car_seen_safety_warning" private const val KEY_SEEN_SAFETY_WARNING = "car_seen_safety_warning"
private const val KEY_UNITS = "car_units" private const val KEY_UNITS = "car_units"
@@ -55,6 +61,9 @@ object CarSettings {
private val openFlow = MutableStateFlow(HOME) private val openFlow = MutableStateFlow(HOME)
private val nightFlow = MutableStateFlow(NightMode.AUTO) private val nightFlow = MutableStateFlow(NightMode.AUTO)
private val pipeAudioFlow = MutableStateFlow(true) private val pipeAudioFlow = MutableStateFlow(true)
private val speakDirectionsFlow = MutableStateFlow(true)
private val bluetoothWarningFlow = MutableStateFlow(true)
private val locationWarningFlow = MutableStateFlow(true)
private val lastNavFlow = MutableStateFlow<String?>(null) private val lastNavFlow = MutableStateFlow<String?>(null)
private val lastMediaFlow = MutableStateFlow<String?>(null) private val lastMediaFlow = MutableStateFlow<String?>(null)
private val autoplayFlow = MutableStateFlow(false) private val autoplayFlow = MutableStateFlow(false)
@@ -66,8 +75,9 @@ object CarSettings {
/** Id of the [GsTheme] controls are drawn in; an unknown id falls back to the default. */ /** Id of the [GsTheme] controls are drawn in; an unknown id falls back to the default. */
val uiTheme: StateFlow<String> = uiThemeFlow val uiTheme: StateFlow<String> = uiThemeFlow
private val pinnedFlow = MutableStateFlow<Set<String>>(emptySet()) private val pinnedFlow = MutableStateFlow<Set<String>>(emptySet())
private val appOrderFlow = MutableStateFlow<List<String>>(emptyList())
private val experimentalFeaturesFlow = MutableStateFlow(false) private val experimentalFeaturesFlow = MutableStateFlow(false)
private val voiceAssistantFlow = MutableStateFlow(false) private val systemSpeechFlow = MutableStateFlow(false)
private val carSensorsFlow = MutableStateFlow(false) private val carSensorsFlow = MutableStateFlow(false)
private val seenSafetyWarningFlow = MutableStateFlow(false) private val seenSafetyWarningFlow = MutableStateFlow(false)
private val unitsFlow = MutableStateFlow(Units.AUTO) private val unitsFlow = MutableStateFlow(Units.AUTO)
@@ -120,6 +130,9 @@ object CarSettings {
/** Flattened component names of the apps long-pressed onto the nav bar as shortcuts. */ /** Flattened component names of the apps long-pressed onto the nav bar as shortcuts. */
val pinnedApps: StateFlow<Set<String>> = pinnedFlow val pinnedApps: StateFlow<Set<String>> = pinnedFlow
/** The launcher's order once the driver has moved a tile; empty keeps the grouped default. */
val appOrder: StateFlow<List<String>> = appOrderFlow
/** Flattened component names of the map and media apps used last, brought back on connect. */ /** Flattened component names of the map and media apps used last, brought back on connect. */
val lastNav: StateFlow<String?> = lastNavFlow val lastNav: StateFlow<String?> = lastNavFlow
val lastMedia: StateFlow<String?> = lastMediaFlow val lastMedia: StateFlow<String?> = lastMediaFlow
@@ -141,22 +154,41 @@ object CarSettings {
* Whether a media app's sound is captured and sent over the car link. On by default, since * Whether a media app's sound is captured and sent over the car link. On by default, since
* sound in the car is the point of the exercise; the platform allows the capture only behind a * sound in the car is the point of the exercise; the platform allows the capture only behind a
* consent dialog, which [app.seb3thehacker.gearslip.audio.CarAudio] raises once the car has * consent dialog, which [app.seb3thehacker.gearslip.audio.CarAudio] raises once the car has
* somewhere to play it. Turned off, the app plays through the phone's normal output instead. * somewhere to play it. Turned off ("Bluetooth" in the settings), the app plays through the
* phone's normal output instead, and Android never asks.
*/ */
val pipeAudio: StateFlow<Boolean> = pipeAudioFlow val pipeAudio: StateFlow<Boolean> = pipeAudioFlow
/** Gearslip reads the map app's turns aloud on the car's directions channel; see DirectionsVoice. */
val speakDirections: StateFlow<Boolean> = speakDirectionsFlow
/** With Bluetooth audio, warn when the phone isn't connected to the car; see BluetoothAudioCheck. */
val bluetoothWarning: StateFlow<Boolean> = bluetoothWarningFlow
/** Warn on the car when the phone's location is off; see LocationCheck. */
val locationWarning: StateFlow<Boolean> = locationWarningFlow
/**
* Whether the driver has already been offered Bluetooth after turning down the capture
* dialog. Offered once only: someone who chose to keep the USB route shouldn't be nagged.
*/
/** False until the driver picks USB or Bluetooth, in setup or in settings. */
val pipeAudioChosen: Boolean
get() = AppSettings.getString(app, KEY_PIPE_AUDIO, "") != ""
val audioChoiceAsked: Boolean
get() = AppSettings.getString(app, KEY_AUDIO_CHOICE_ASKED, "false") == "true"
fun markAudioChoiceAsked() = AppSettings.putString(app, KEY_AUDIO_CHOICE_ASKED, "true")
/** /**
* Off by default: a driver has to opt into experimental features as a whole before any of * Off by default: a driver has to opt into experimental features as a whole before any of
* them - the voice assistant included - show up to turn on individually. * them - the voice assistant included - show up to turn on individually.
*/ */
val experimentalFeaturesEnabled: StateFlow<Boolean> = experimentalFeaturesFlow val experimentalFeaturesEnabled: StateFlow<Boolean> = experimentalFeaturesFlow
/** /** Listen with the phone's own speech service instead of Gearslip's on-device model. */
* The voice assistant is experimental and off by default; a driver has to turn it on val systemSpeech: StateFlow<Boolean> = systemSpeechFlow
* knowingly, from the settings screen, after reading the warning shown there. This flag
* alone does not run the assistant - [experimentalFeaturesEnabled] must also be on.
*/
val voiceAssistantEnabled: StateFlow<Boolean> = voiceAssistantFlow
/** /**
* Vehicle data is experimental and off by default: the launcher's tile for it, and the * Vehicle data is experimental and off by default: the launcher's tile for it, and the
@@ -192,13 +224,20 @@ object CarSettings {
autoplayFlow.value = AppSettings.getString(app, KEY_AUTOPLAY, "false") == "true" autoplayFlow.value = AppSettings.getString(app, KEY_AUTOPLAY, "false") == "true"
playerTitleFlow.value = AppSettings.getString(app, KEY_PLAYER_TITLE, "false") == "true" playerTitleFlow.value = AppSettings.getString(app, KEY_PLAYER_TITLE, "false") == "true"
pipeAudioFlow.value = AppSettings.getString(app, KEY_PIPE_AUDIO, "true") == "true" pipeAudioFlow.value = AppSettings.getString(app, KEY_PIPE_AUDIO, "true") == "true"
// Until the driver sets it, on for USB audio and off for Bluetooth, where the map app's
// own voice already reaches the car.
speakDirectionsFlow.value = AppSettings.getString(app, KEY_SPEAK_DIRECTIONS, pipeAudioFlow.value.toString()) == "true"
bluetoothWarningFlow.value = AppSettings.getString(app, KEY_BLUETOOTH_WARNING, "true") == "true"
locationWarningFlow.value = AppSettings.getString(app, KEY_LOCATION_WARNING, "true") == "true"
nightFlow.value = runCatching { nightFlow.value = runCatching {
NightMode.valueOf(AppSettings.getString(app, KEY_NIGHT, NightMode.AUTO.name)) NightMode.valueOf(AppSettings.getString(app, KEY_NIGHT, NightMode.AUTO.name))
}.getOrDefault(NightMode.AUTO) }.getOrDefault(NightMode.AUTO)
pinnedFlow.value = AppSettings.getString(app, KEY_PINNED, "") pinnedFlow.value = AppSettings.getString(app, KEY_PINNED, "")
.split(",").filter { it.isNotEmpty() }.toSet() .split(",").filter { it.isNotEmpty() }.toSet()
appOrderFlow.value = AppSettings.getString(app, KEY_APP_ORDER, "")
.split(",").filter { it.isNotEmpty() }
experimentalFeaturesFlow.value = AppSettings.getString(app, KEY_EXPERIMENTAL_FEATURES, "false") == "true" experimentalFeaturesFlow.value = AppSettings.getString(app, KEY_EXPERIMENTAL_FEATURES, "false") == "true"
voiceAssistantFlow.value = AppSettings.getString(app, KEY_VOICE_ASSISTANT, "false") == "true" systemSpeechFlow.value = AppSettings.getString(app, KEY_SYSTEM_SPEECH, "false") == "true"
carSensorsFlow.value = AppSettings.getString(app, KEY_CAR_SENSORS, "false") == "true" carSensorsFlow.value = AppSettings.getString(app, KEY_CAR_SENSORS, "false") == "true"
seenSafetyWarningFlow.value = AppSettings.getString(app, KEY_SEEN_SAFETY_WARNING, "false") == "true" seenSafetyWarningFlow.value = AppSettings.getString(app, KEY_SEEN_SAFETY_WARNING, "false") == "true"
unitsFlow.value = runCatching { unitsFlow.value = runCatching {
@@ -248,9 +287,32 @@ object CarSettings {
AppSettings.putString(app, KEY_AUTOPLAY, on.toString()) AppSettings.putString(app, KEY_AUTOPLAY, on.toString())
} }
/**
* Takes effect at once: switching to Bluetooth mid-drive stops the capture, and switching
* back asks for it again (or waits for the car, if none is connected).
*/
fun setPipeAudio(on: Boolean) { fun setPipeAudio(on: Boolean) {
pipeAudioFlow.value = on // Saved even when unchanged, so picking the default still counts as a choice.
AppSettings.putString(app, KEY_PIPE_AUDIO, on.toString()) AppSettings.putString(app, KEY_PIPE_AUDIO, on.toString())
if (pipeAudioFlow.value == on) return
pipeAudioFlow.value = on
if (AppSettings.getString(app, KEY_SPEAK_DIRECTIONS, "") == "") speakDirectionsFlow.value = on
if (on) CarAudio.request() else CarAudio.release()
}
fun setSpeakDirections(on: Boolean) {
speakDirectionsFlow.value = on
AppSettings.putString(app, KEY_SPEAK_DIRECTIONS, on.toString())
}
fun setBluetoothWarning(on: Boolean) {
bluetoothWarningFlow.value = on
AppSettings.putString(app, KEY_BLUETOOTH_WARNING, on.toString())
}
fun setLocationWarning(on: Boolean) {
locationWarningFlow.value = on
AppSettings.putString(app, KEY_LOCATION_WARNING, on.toString())
} }
fun setExperimentalFeaturesEnabled(on: Boolean) { fun setExperimentalFeaturesEnabled(on: Boolean) {
@@ -258,9 +320,9 @@ object CarSettings {
AppSettings.putString(app, KEY_EXPERIMENTAL_FEATURES, on.toString()) AppSettings.putString(app, KEY_EXPERIMENTAL_FEATURES, on.toString())
} }
fun setVoiceAssistantEnabled(on: Boolean) { fun setSystemSpeech(on: Boolean) {
voiceAssistantFlow.value = on systemSpeechFlow.value = on
AppSettings.putString(app, KEY_VOICE_ASSISTANT, on.toString()) AppSettings.putString(app, KEY_SYSTEM_SPEECH, on.toString())
} }
fun setCarSensorsEnabled(on: Boolean) { fun setCarSensorsEnabled(on: Boolean) {
@@ -286,4 +348,9 @@ object CarSettings {
pinnedFlow.value = next pinnedFlow.value = next
AppSettings.putString(app, KEY_PINNED, next.joinToString(",")) AppSettings.putString(app, KEY_PINNED, next.joinToString(","))
} }
fun setAppOrder(ids: List<String>) {
appOrderFlow.value = ids
AppSettings.putString(app, KEY_APP_ORDER, ids.joinToString(","))
}
} }
@@ -53,13 +53,16 @@ fun CarSettingsScreen() {
val units by CarSettings.units.collectAsState() val units by CarSettings.units.collectAsState()
val whatsNew by CarSettings.whatsNewAfterUpdates.collectAsState() val whatsNew by CarSettings.whatsNewAfterUpdates.collectAsState()
val pipeAudio by CarSettings.pipeAudio.collectAsState() val pipeAudio by CarSettings.pipeAudio.collectAsState()
val speakDirections by CarSettings.speakDirections.collectAsState()
val bluetoothWarning by CarSettings.bluetoothWarning.collectAsState()
val locationWarning by CarSettings.locationWarning.collectAsState()
val autoplay by CarSettings.autoplay.collectAsState() val autoplay by CarSettings.autoplay.collectAsState()
val playerTitle by CarSettings.playerTitle.collectAsState() val playerTitle by CarSettings.playerTitle.collectAsState()
val theme by CarSettings.appTheme.collectAsState() val theme by CarSettings.appTheme.collectAsState()
val uiTheme by CarSettings.uiTheme.collectAsState() val uiTheme by CarSettings.uiTheme.collectAsState()
val experimentalFeatures by CarSettings.experimentalFeaturesEnabled.collectAsState() val experimentalFeatures by CarSettings.experimentalFeaturesEnabled.collectAsState()
val voiceAssistant by CarSettings.voiceAssistantEnabled.collectAsState()
val carSensors by CarSettings.carSensorsEnabled.collectAsState() val carSensors by CarSettings.carSensorsEnabled.collectAsState()
val systemSpeech by CarSettings.systemSpeech.collectAsState()
val display by CarEnvironment.display.collectAsState() val display by CarEnvironment.display.collectAsState()
val vehicle by CarEnvironment.vehicle.collectAsState() val vehicle by CarEnvironment.vehicle.collectAsState()
val context = LocalContext.current val context = LocalContext.current
@@ -74,7 +77,7 @@ fun CarSettingsScreen() {
.padding(horizontal = 24.dp, vertical = 16.dp), .padding(horizontal = 24.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(20.dp), verticalArrangement = Arrangement.spacedBy(20.dp),
) { ) {
Text("Settings", style = MaterialTheme.typography.headlineMedium, fontWeight = FontWeight.Bold) ScreenHeader("Settings")
ChoiceRow( ChoiceRow(
"Open on connect", "Open on connect",
@@ -125,18 +128,49 @@ fun CarSettingsScreen() {
) { CarSettings.setAutoplay(it) } ) { CarSettings.setAutoplay(it) }
ChoiceRow( ChoiceRow(
"Media audio", "Car audio",
listOf(false to "Phone output", true to "Through Gearslip"), listOf(false to "Bluetooth", true to "USB"),
pipeAudio, pipeAudio,
) { CarSettings.setPipeAudio(it) } ) { CarSettings.setPipeAudio(it) }
Text( Text(
"Through Gearslip sends the sound over the USB link. Android asks for consent the " + "USB sends the phone's sound through Gearslip. Android asks to share your screen each " +
"first time a media app plays in the car, and some apps refuse to be captured at " + "drive, and a few apps block it. Bluetooth plays as the phone normally would, and " +
"all. Phone output needs no permission: the app plays as it normally would, for " + "Android never asks.",
"example over Bluetooth to the car.",
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
if (!pipeAudio) {
ChoiceRow(
"Warn when Bluetooth isn't connected",
listOf(false to "Off", true to "On"),
bluetoothWarning,
) { CarSettings.setBluetoothWarning(it) }
}
ChoiceRow(
"Warn when location is off",
listOf(false to "Off", true to "On"),
locationWarning,
) { CarSettings.setLocationWarning(it) }
ChoiceRow(
"Spoken directions",
listOf(false to "Off", true to "On"),
speakDirections,
) { CarSettings.setSpeakDirections(it) }
Text(
"Gearslip reads each turn aloud through the car's speakers, and the music dips while it " +
"speaks. Turn it off if your map app's own voice already plays in the car.",
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
)
ExperimentalFeatureRow(
title = "Use the phone's speech recognition",
description = "Listens through the phone's own speech service instead of Gearslip's. On " +
"most phones that's Google's, which may send your voice to Google. GrapheneOS has " +
"none, so Gearslip keeps using its own.",
checked = systemSpeech,
) { CarSettings.setSystemSpeech(it) }
WebPageSetting() WebPageSetting()
@@ -148,7 +182,7 @@ fun CarSettingsScreen() {
experimentalFeatures, experimentalFeatures,
) { CarSettings.setExperimentalFeaturesEnabled(it) } ) { CarSettings.setExperimentalFeaturesEnabled(it) }
if (experimentalFeatures) { if (experimentalFeatures) {
ExperimentalFeaturesList(voiceAssistant, carSensors) ExperimentalFeaturesList(carSensors)
} }
Column(verticalArrangement = Arrangement.spacedBy(8.dp)) { Column(verticalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -205,21 +239,11 @@ private fun <T> ChoiceRow(title: String, options: List<Pair<T, String>>, selecte
* so the list reads fine whether there's one entry or several. * so the list reads fine whether there's one entry or several.
*/ */
@Composable @Composable
private fun ExperimentalFeaturesList(voiceAssistant: Boolean, carSensors: Boolean) { private fun ExperimentalFeaturesList(carSensors: Boolean) {
Column( Column(
Modifier.fillMaxWidth(), Modifier.fillMaxWidth(),
verticalArrangement = Arrangement.spacedBy(10.dp), verticalArrangement = Arrangement.spacedBy(10.dp),
) { ) {
ExperimentalFeatureRow(
title = "Voice assistant",
description = "Push-to-talk and the car's voice button start listening: navigate, " +
"play media, call, or open an app by saying so. Can mishear you or act on the " +
"wrong command - keep your eyes on the road, and confirm anything it does " +
"before you rely on it.",
warning = true,
checked = voiceAssistant,
) { CarSettings.setVoiceAssistantEnabled(it) }
ExperimentalFeatureRow( ExperimentalFeatureRow(
title = "Vehicle data", title = "Vehicle data",
description = "Adds a Vehicle data tile: a live, raw dump of the head unit's sensor " + description = "Adds a Vehicle data tile: a live, raw dump of the head unit's sensor " +
@@ -265,7 +289,7 @@ private fun ExperimentalFeatureRow(
} }
} }
/** The URL (or raw HTML) the Web app shows. Edited with [CarKeyboard] - no phone in hand needed. */ /** The URL (or raw HTML) the Browser app shows. Edited with [CarKeyboard] - no phone in hand needed. */
@Composable @Composable
private fun WebPageSetting() { private fun WebPageSetting() {
val context = LocalContext.current val context = LocalContext.current
@@ -290,7 +314,7 @@ private fun WebPageSetting() {
) )
} }
Text( Text(
"Shown by the Web app on the launcher. Applies on the next connection.", "Shown by the Browser app on the launcher. Applies on the next connection.",
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
@@ -0,0 +1,62 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.platform.LocalAccessibilityManager
import androidx.compose.ui.semantics.LiveRegionMode
import androidx.compose.ui.semantics.liveRegion
import androidx.compose.ui.semantics.semantics
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.withTimeoutOrNull
/** Draw inside the encoded car display; a normal Android Toast would appear on the phone. */
@Composable
internal fun CarToastOverlay(modifier: Modifier = Modifier) {
val current by CarToasts.message.collectAsState()
val message = current ?: return
val accessibility = LocalAccessibilityManager.current
LaunchedEffect(message, accessibility) {
val timeout = accessibility?.calculateRecommendedTimeoutMillis(
message.durationMs, containsIcons = false, containsText = true, containsControls = false,
) ?: message.durationMs
// The full time, unless another toast is waiting: then it gives way after the minimum.
delay(minOf(timeout, CarToasts.MIN_SHOWN_MS))
withTimeoutOrNull((timeout - CarToasts.MIN_SHOWN_MS).coerceAtLeast(0)) {
CarToasts.waiting.first { it > 0 }
}
CarToasts.dismiss(message)
}
// A non-interactive overlay leaves the map's touch handling and focus intact.
Box(
modifier
.padding(horizontal = 24.dp, vertical = 16.dp)
.widthIn(max = 560.dp)
.clip(RoundedCornerShape(20.dp))
.background(MaterialTheme.colorScheme.surfaceContainerHigh)
.semantics { liveRegion = LiveRegionMode.Polite },
) {
Text(
message.text,
modifier = Modifier.padding(horizontal = 20.dp, vertical = 12.dp),
color = MaterialTheme.colorScheme.onSurface,
style = MaterialTheme.typography.titleMedium,
maxLines = 3,
overflow = TextOverflow.Ellipsis,
)
}
}
@@ -0,0 +1,65 @@
package app.seb3thehacker.gearslip.car
import androidx.annotation.MainThread
import androidx.car.app.CarToast
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* One toast at a time across navigation and browse apps. A newer one waits its turn instead of
* replacing the one on screen, which gets at least [MIN_SHOWN_MS] before it gives way.
*/
@MainThread
internal object CarToasts {
// Identity matters: repeating identical text must restart its display timer.
class Message(val owner: Any, val text: String, val durationMs: Long)
/** Long enough to read a short toast at a glance; a waiting toast cuts the rest short. */
const val MIN_SHOWN_MS = 2_000L
/** A burst beyond this many is stale by the time it would show; the oldest waiting go first. */
private const val MAX_WAITING = 3
private val current = MutableStateFlow<Message?>(null)
val message: StateFlow<Message?> = current.asStateFlow()
private val queue = ArrayDeque<Message>()
private val _waiting = MutableStateFlow(0)
val waiting: StateFlow<Int> = _waiting.asStateFlow()
fun show(owner: Any, text: String, duration: Int) {
if (text.isBlank()) return
// Longer than a phone toast: the driver only glances at the screen now and then.
val durationMs = if (duration == CarToast.LENGTH_LONG) 7_000L else 4_000L
val next = Message(owner, text, durationMs)
val shown = current.value
when {
shown == null -> current.value = next
// The same words again restart the one on screen rather than queueing a copy.
shown.text == text -> current.value = next
queue.lastOrNull()?.text == text -> Unit
else -> {
queue.addLast(next)
while (queue.size > MAX_WAITING) queue.removeFirst()
}
}
_waiting.value = queue.size
}
fun dismiss(message: Message) {
// An old timer must not dismiss a newer toast, even if its text is identical.
if (current.value === message) advance()
}
fun clear(owner: Any) {
// Disconnecting the map must not clear a toast belonging to the browse app.
queue.removeAll { it.owner === owner }
if (current.value?.owner === owner) advance() else _waiting.value = queue.size
}
private fun advance() {
current.value = queue.removeFirstOrNull()
_waiting.value = queue.size
}
}
@@ -2,6 +2,7 @@ package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.Image import androidx.compose.foundation.Image
import androidx.compose.foundation.clickable import androidx.compose.foundation.clickable
import androidx.compose.foundation.LocalIndication
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.BoxWithConstraints import androidx.compose.foundation.layout.BoxWithConstraints
@@ -43,6 +44,10 @@ import androidx.compose.material3.dynamicDarkColorScheme
import androidx.compose.material3.dynamicLightColorScheme import androidx.compose.material3.dynamicLightColorScheme
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.CompositionLocalProvider import androidx.compose.runtime.CompositionLocalProvider
import androidx.compose.ui.hapticfeedback.HapticFeedback
import androidx.compose.ui.hapticfeedback.HapticFeedbackType
import androidx.compose.ui.platform.LocalHapticFeedback
import androidx.compose.ui.platform.LocalView
import androidx.compose.runtime.State import androidx.compose.runtime.State
import androidx.compose.runtime.collectAsState import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
@@ -120,8 +125,16 @@ fun CarUi() {
controlsWake.tick() controlsWake.tick()
} }
} }
// The knob counts as a touch: the map's buttons can't be found while they're faded out.
LaunchedEffect(controlsWake) { CarKeys.focusMoves.collect { controlsWake.touch() } }
remember(appContext) { Prefetch.warm(appContext, force = true); CarServices.init(appContext) } remember(appContext) { Prefetch.warm(appContext, force = true); CarServices.init(appContext) }
DisposableEffect(Unit) { onDispose { CarServices.shutdown() } } DisposableEffect(Unit) { onDispose { CarServices.shutdown() } }
// Ask for the phone's sound as soon as the car connects, not only when a media app opens:
// a video in the web browser or a map's voice needs it too. Asks once, after the car offers
// somewhere to play it.
LaunchedEffect(Unit) { if (CarSettings.pipeAudio.value) app.seb3thehacker.gearslip.audio.CarAudio.request() }
// A new version gets one card in the top right, a few seconds in and only while parked.
LaunchedEffect(Unit) { app.seb3thehacker.gearslip.update.UpdateChecker.onCarConnected(appContext) }
// The map follows the light outside, whatever the app's own theme is set to. // The map follows the light outside, whatever the app's own theme is set to.
LaunchedEffect(darkOutside) { CarServices.nav.pushConfiguration() } LaunchedEffect(darkOutside) { CarServices.nav.pushConfiguration() }
@@ -146,14 +159,27 @@ fun CarUi() {
VoiceReply.state.value.phase != VoiceReply.Phase.IDLE -> VoiceReply.cancel() VoiceReply.state.value.phase != VoiceReply.Phase.IDLE -> VoiceReply.cancel()
else -> navigator.back() else -> navigator.back()
} }
CarKeys.ScreenKey.HOME, CarKeys.ScreenKey.MAP -> navigator.home() // Home is the dashboard, over the map, so Back from it lands on the map.
CarKeys.ScreenKey.MEDIA -> if (CarServices.mediaApp.value != null) navigator.media() else navigator.apps() CarKeys.ScreenKey.HOME -> { navigator.home(); navigator.dashboard() }
CarKeys.ScreenKey.MAP -> navigator.home()
CarKeys.ScreenKey.MEDIA ->
if (CarServices.mediaApp.value != null || CarServices.reopenLastMedia()) navigator.media()
else navigator.apps()
CarKeys.ScreenKey.PHONE -> BuiltInApps.Phone.open(navigator) CarKeys.ScreenKey.PHONE -> BuiltInApps.Phone.open(navigator)
} }
} }
} }
// The car screen is drawn on the phone, so every long press or text-handle drag would buzz
// the phone in its mount. Nobody feels that while driving; it's only noise.
val view = LocalView.current
DisposableEffect(view) {
view.isHapticFeedbackEnabled = false
onDispose { }
}
CompositionLocalProvider( CompositionLocalProvider(
LocalHapticFeedback provides NoHaptics,
LocalDensity provides Density(base.density * scale, base.fontScale), LocalDensity provides Density(base.density * scale, base.fontScale),
LocalCarNavigator provides navigator, LocalCarNavigator provides navigator,
LocalDarkOutside provides darkOutside, LocalDarkOutside provides darkOutside,
@@ -161,6 +187,7 @@ fun CarUi() {
LocalGsTheme provides GsThemes.byId(uiThemeId), LocalGsTheme provides GsThemes.byId(uiThemeId),
) { ) {
MaterialTheme(colorScheme = colors) { MaterialTheme(colorScheme = colors) {
CompositionLocalProvider(LocalIndication provides FocusRingIndication(MaterialTheme.colorScheme.primary)) {
Surface( Surface(
Modifier Modifier
.fillMaxSize() .fillMaxSize()
@@ -220,6 +247,9 @@ fun CarUi() {
} }
} }
} }
CarToastOverlay(Modifier.align(Alignment.BottomCenter))
CallPromptOverlay()
BluetoothPairOverlay()
PopupTimer() PopupTimer()
// The dashboard is already listing them, so it needs no popup. // The dashboard is already listing them, so it needs no popup.
NotificationPopup( NotificationPopup(
@@ -227,14 +257,15 @@ fun CarUi() {
modifier = Modifier.align(Alignment.TopEnd), modifier = Modifier.align(Alignment.TopEnd),
) )
// Above the popup: a call is more urgent than any notification. // Above the popup: a call is more urgent than any notification.
VoiceHelpOverlay(Modifier.align(Alignment.Center))
CallOverlay(Modifier.align(Alignment.TopCenter)) CallOverlay(Modifier.align(Alignment.TopCenter))
AssistantOverlay(Modifier.align(Alignment.TopCenter)) AssistantOverlay(Modifier.align(Alignment.BottomCenter))
if (app.seb3thehacker.gearslip.BuildConfig.DEBUG) { if (app.seb3thehacker.gearslip.BuildConfig.DEBUG) {
// Bottom corner, not top: a screen's own header almost always puts // Bottom corner, not top: a screen's own header almost always puts
// something in the top-right (settings, search, a Liked Songs // something in the top-right (settings, search, a Liked Songs
// shortcut) and this badge used to sit right on top of it, hiding // shortcut) and this badge used to sit right on top of it, hiding
// whatever the screen drew there. // whatever the screen drew there.
DevBuildBadge(Modifier.align(Alignment.BottomEnd)) DebugBuildBadge(Modifier.align(Alignment.BottomEnd))
} }
} }
// The keyboard takes the nav bar's row instead of squeezing in above it - // The keyboard takes the nav bar's row instead of squeezing in above it -
@@ -248,6 +279,7 @@ fun CarUi() {
if (safetyWarningSeen) CarWhatsNew() if (safetyWarningSeen) CarWhatsNew()
} }
} }
}
} }
} }
} }
@@ -262,13 +294,15 @@ fun CarUi() {
@Composable @Composable
private fun CarSafetyWizard(onDone: () -> Unit) { private fun CarSafetyWizard(onDone: () -> Unit) {
Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) { Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
// The card and its button share one width, capped so neither sprawls across a wide screen.
Column( Column(
Modifier.fillMaxWidth().padding(24.dp), Modifier.widthIn(max = 640.dp).fillMaxWidth().padding(24.dp),
horizontalAlignment = Alignment.CenterHorizontally, horizontalAlignment = Alignment.CenterHorizontally,
) { ) {
Surface( Surface(
color = MaterialTheme.colorScheme.surfaceContainerLow, color = MaterialTheme.colorScheme.surfaceContainerLow,
shape = MaterialTheme.shapes.extraLarge, shape = MaterialTheme.shapes.extraLarge,
modifier = Modifier.fillMaxWidth(),
) { ) {
Column(Modifier.padding(24.dp), verticalArrangement = Arrangement.spacedBy(12.dp)) { Column(Modifier.padding(24.dp), verticalArrangement = Arrangement.spacedBy(12.dp)) {
Text("Drive safely", style = ChromeType.headline, fontWeight = FontWeight.Bold) Text("Drive safely", style = ChromeType.headline, fontWeight = FontWeight.Bold)
@@ -348,7 +382,7 @@ private fun AppWontStart(app: String, onBack: () -> Unit) {
* that matters while driving, since a debug build is slower and less tested than what ships. * that matters while driving, since a debug build is slower and less tested than what ships.
*/ */
@Composable @Composable
private fun DevBuildBadge(modifier: Modifier = Modifier) { private fun DebugBuildBadge(modifier: Modifier = Modifier) {
Surface( Surface(
modifier = modifier.padding(6.dp), modifier = modifier.padding(6.dp),
color = MaterialTheme.colorScheme.error, color = MaterialTheme.colorScheme.error,
@@ -356,7 +390,7 @@ private fun DevBuildBadge(modifier: Modifier = Modifier) {
shape = MaterialTheme.shapes.small, shape = MaterialTheme.shapes.small,
) { ) {
Text( Text(
"DEV BUILD", "DEBUG BUILD",
style = MaterialTheme.typography.labelSmall, style = MaterialTheme.typography.labelSmall,
fontWeight = FontWeight.Bold, fontWeight = FontWeight.Bold,
modifier = Modifier.padding(horizontal = 8.dp, vertical = 3.dp), modifier = Modifier.padding(horizontal = 8.dp, vertical = 3.dp),
@@ -437,12 +471,8 @@ private fun CarNavBar(navigator: CarNavigator) {
NavHomeButton(navigator, entries, running) NavHomeButton(navigator, entries, running)
Spacer(Modifier.width(6.dp)) Spacer(Modifier.width(6.dp))
NavItem(MediaIcons.Apps, navigator.current == CarScreen.Apps) { navigator.apps() } NavItem(MediaIcons.Apps, navigator.current == CarScreen.Apps) { navigator.apps() }
val experimentalFeatures by CarSettings.experimentalFeaturesEnabled.collectAsState() Spacer(Modifier.width(6.dp))
val voiceEnabled by CarSettings.voiceAssistantEnabled.collectAsState() NavAssistantIcon()
if (experimentalFeatures && voiceEnabled) {
Spacer(Modifier.width(6.dp))
NavAssistantIcon()
}
// Pinned shortcuts, then whatever else is open right now. A pinned app that's also // Pinned shortcuts, then whatever else is open right now. A pinned app that's also
// open stays in its pinned spot, marked with a dot, instead of showing up twice. // open stays in its pinned spot, marked with a dot, instead of showing up twice.
@@ -873,3 +903,8 @@ private fun ChargingGlyph(modifier: Modifier = Modifier, tint: Color = MaterialT
drawPath(bolt, tint) drawPath(bolt, tint)
} }
} }
/** Stands in for the phone's vibration motor on the car screen: does nothing. */
private object NoHaptics : HapticFeedback {
override fun performHapticFeedback(hapticFeedbackType: HapticFeedbackType) {}
}
@@ -0,0 +1,199 @@
package app.seb3thehacker.gearslip.car
import androidx.car.app.model.CarText
import androidx.car.app.model.Distance
import androidx.car.app.navigation.model.Maneuver
import androidx.car.app.navigation.model.NavigationTemplate
import androidx.car.app.navigation.model.RoutingInfo
import androidx.car.app.navigation.model.Step
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.Protobuf
/**
* The next turn on the car's instrument cluster, over its NavigationStatusService (Service
* field 8), the way Android Auto feeds it:
*
* -> NavigationStatus { status = 1 (1 active, 2 inactive, 3 rerouting) }
* -> NavigationNextTurnEvent { road = 1; turn_side = 2; event = 3; turn_number = 5; turn_angle = 6 }
* -> NavigationNextTurnDistanceEvent { distance_meters = 1; time_to_turn_seconds = 2;
* display_distance_e3 = 3; display_distance_unit = 4 }
*
* Newer cars also take a NavigationState message; the deprecated pair above is what a 2018
* Uconnect understands. The turn comes from the map app's trip (what the Car App Library hands
* to clusters), or from its routing card when it sends no trip.
*/
class ClusterNav(
val channelId: Int,
private val sendOnChannel: (messageId: Int, body: ByteArray) -> Unit,
) {
@Volatile private var running = false
private var sentStatus = -1
private var sentTurn: String? = null
private var sentDistance: String? = null
fun onMessage(messageId: Int, body: ByteArray) {
when (messageId) {
MSG_CHANNEL_OPEN_RESPONSE -> {
val status = Protobuf.readInt32Field(body, 1) ?: 0
GearslipLog.i("cluster: <- ChannelOpenResponse: status=$status")
if (status == 0) start()
}
// The car saying when it wants turns. Sending all along is harmless, so these are only logged.
MSG_START -> GearslipLog.i("cluster: <- start")
MSG_STOP -> GearslipLog.i("cluster: <- stop")
else -> GearslipLog.i("cluster: <- unhandled message id=$messageId (${body.size} bytes)")
}
}
private fun start() {
if (running) return
running = true
Thread({
while (running) {
runCatching { tick() }.onFailure { GearslipLog.w("cluster: ${it.message}") }
Thread.sleep(TICK_MS)
}
}, "gearslip-cluster").start()
}
fun stop() {
running = false
}
private fun tick() {
val turn = NavTurn.current()
val status = when {
turn == null -> STATUS_INACTIVE
turn.rerouting -> STATUS_REROUTING
else -> STATUS_ACTIVE
}
if (status != sentStatus) {
sendOnChannel(MSG_STATUS, Protobuf.varintField(1, status.toLong()))
GearslipLog.i("cluster: -> ${STATUS_NAMES[status]}")
sentStatus = status
if (status == STATUS_INACTIVE) { sentTurn = null; sentDistance = null }
}
if (turn == null || status == STATUS_REROUTING) return
sendTurn(turn.step)
turn.distance?.let { sendDistance(it, turn.seconds) }
}
private fun sendTurn(step: Step) {
val maneuver = step.maneuver
val (event, side) = maneuver?.let { eventFor(it.type) } ?: (EVENT_UNKNOWN to SIDE_UNSPECIFIED)
val road = step.road.text().ifBlank { step.cue.text() }
val key = "$road|$event|$side|${maneuver?.roundaboutExitNumber}"
if (key == sentTurn) return
sentTurn = key
var body = Protobuf.stringField(1, road) +
Protobuf.varintField(2, side.toLong()) +
Protobuf.varintField(3, event.toLong())
maneuver?.roundaboutExitNumber?.takeIf { it > 0 }?.let { body += Protobuf.varintField(5, it.toLong()) }
maneuver?.roundaboutExitAngle?.takeIf { it > 0 }?.let { body += Protobuf.varintField(6, it.toLong()) }
sendOnChannel(MSG_TURN, body)
GearslipLog.i("cluster: -> turn event=$event side=$side road=\"$road\"")
}
/**
* The Car App Library's distance units use the same numbers as the protocol's (1 metres ...
* 7 yards), so the app's own display choice passes straight through.
*/
private fun sendDistance(distance: Distance, seconds: Long?) {
val meters = NavTurn.toMeters(distance)
val key = "${distance.displayDistance}|${distance.displayUnit}|${seconds ?: -1}"
if (key == sentDistance) return
sentDistance = key
sendOnChannel(
MSG_DISTANCE,
Protobuf.varintField(1, meters.toLong()) +
Protobuf.varintField(2, (seconds ?: 0L).coerceAtLeast(0)) +
Protobuf.varintField(3, (distance.displayDistance * 1000).toLong()) +
Protobuf.varintField(4, distance.displayUnit.toLong()),
)
}
private fun CarText?.text() = this?.toString().orEmpty()
companion object {
private const val TICK_MS = 500L
private const val MSG_CHANNEL_OPEN_RESPONSE = 8
private const val MSG_START = 32769
private const val MSG_STOP = 32770
private const val MSG_STATUS = 32771
private const val MSG_TURN = 32772
private const val MSG_DISTANCE = 32773
private const val STATUS_ACTIVE = 1
private const val STATUS_INACTIVE = 2
private const val STATUS_REROUTING = 3
private val STATUS_NAMES = mapOf(1 to "navigating", 2 to "not navigating", 3 to "rerouting")
private const val SIDE_LEFT = 1
private const val SIDE_RIGHT = 2
private const val SIDE_UNSPECIFIED = 3
private const val EVENT_UNKNOWN = 0
/** The car's turn icon and side for a Car App Library manoeuvre. */
fun eventFor(type: Int): Pair<Int, Int> = when (type) {
Maneuver.TYPE_DEPART -> 1 to SIDE_UNSPECIFIED
Maneuver.TYPE_NAME_CHANGE -> 2 to SIDE_UNSPECIFIED
Maneuver.TYPE_KEEP_LEFT, Maneuver.TYPE_TURN_SLIGHT_LEFT -> 3 to SIDE_LEFT
Maneuver.TYPE_KEEP_RIGHT, Maneuver.TYPE_TURN_SLIGHT_RIGHT -> 3 to SIDE_RIGHT
Maneuver.TYPE_TURN_NORMAL_LEFT -> 4 to SIDE_LEFT
Maneuver.TYPE_TURN_NORMAL_RIGHT -> 4 to SIDE_RIGHT
Maneuver.TYPE_TURN_SHARP_LEFT -> 5 to SIDE_LEFT
Maneuver.TYPE_TURN_SHARP_RIGHT -> 5 to SIDE_RIGHT
Maneuver.TYPE_U_TURN_LEFT -> 6 to SIDE_LEFT
Maneuver.TYPE_U_TURN_RIGHT -> 6 to SIDE_RIGHT
Maneuver.TYPE_ON_RAMP_SLIGHT_LEFT, Maneuver.TYPE_ON_RAMP_NORMAL_LEFT, Maneuver.TYPE_ON_RAMP_SHARP_LEFT,
Maneuver.TYPE_ON_RAMP_U_TURN_LEFT -> 7 to SIDE_LEFT
Maneuver.TYPE_ON_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_ON_RAMP_NORMAL_RIGHT, Maneuver.TYPE_ON_RAMP_SHARP_RIGHT,
Maneuver.TYPE_ON_RAMP_U_TURN_RIGHT -> 7 to SIDE_RIGHT
Maneuver.TYPE_OFF_RAMP_SLIGHT_LEFT, Maneuver.TYPE_OFF_RAMP_NORMAL_LEFT -> 8 to SIDE_LEFT
Maneuver.TYPE_OFF_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_OFF_RAMP_NORMAL_RIGHT -> 8 to SIDE_RIGHT
Maneuver.TYPE_FORK_LEFT -> 9 to SIDE_LEFT
Maneuver.TYPE_FORK_RIGHT -> 9 to SIDE_RIGHT
Maneuver.TYPE_MERGE_LEFT -> 10 to SIDE_LEFT
Maneuver.TYPE_MERGE_RIGHT -> 10 to SIDE_RIGHT
Maneuver.TYPE_MERGE_SIDE_UNSPECIFIED -> 10 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_ENTER_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_CCW -> 11 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_EXIT_CCW -> 12 to SIDE_UNSPECIFIED
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW_WITH_ANGLE,
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW_WITH_ANGLE -> 13 to SIDE_UNSPECIFIED
Maneuver.TYPE_STRAIGHT -> 14 to SIDE_UNSPECIFIED
Maneuver.TYPE_FERRY_BOAT, Maneuver.TYPE_FERRY_BOAT_LEFT, Maneuver.TYPE_FERRY_BOAT_RIGHT -> 16 to SIDE_UNSPECIFIED
Maneuver.TYPE_FERRY_TRAIN, Maneuver.TYPE_FERRY_TRAIN_LEFT, Maneuver.TYPE_FERRY_TRAIN_RIGHT -> 17 to SIDE_UNSPECIFIED
Maneuver.TYPE_DESTINATION, Maneuver.TYPE_DESTINATION_STRAIGHT -> 19 to SIDE_UNSPECIFIED
Maneuver.TYPE_DESTINATION_LEFT -> 19 to SIDE_LEFT
Maneuver.TYPE_DESTINATION_RIGHT -> 19 to SIDE_RIGHT
else -> EVENT_UNKNOWN to SIDE_UNSPECIFIED
}
}
}
/** One turn as the car wants it, from whichever source the map app fills. */
internal class NavTurn(val step: Step, val distance: Distance?, val seconds: Long?, val rerouting: Boolean) {
companion object {
/** The map app's next turn: from its trip (what clusters get), or its routing card. */
fun current(): NavTurn? {
val nav = CarServices.navOrNull() ?: return null
nav.trip.value?.let { trip ->
val step = trip.steps.firstOrNull() ?: return@let
val estimate = trip.stepTravelEstimates.firstOrNull()
return NavTurn(step, estimate?.remainingDistance, estimate?.remainingTimeSeconds?.takeIf { it >= 0 }, trip.isLoading)
}
val info = (nav.template.value as? NavigationTemplate)?.navigationInfo as? RoutingInfo ?: return null
val step = info.currentStep ?: return null
return NavTurn(step, info.currentDistance, null, info.isLoading)
}
fun toMeters(d: Distance): Int = (d.displayDistance * when (d.displayUnit) {
Distance.UNIT_KILOMETERS, Distance.UNIT_KILOMETERS_P1 -> 1000.0
Distance.UNIT_MILES, Distance.UNIT_MILES_P1 -> 1609.344
Distance.UNIT_FEET -> 0.3048
Distance.UNIT_YARDS -> 0.9144
else -> 1.0
}).toInt()
}
}
@@ -0,0 +1,326 @@
package app.seb3thehacker.gearslip.car
import android.content.Context
import android.os.Bundle
import android.speech.tts.TextToSpeech
import android.speech.tts.UtteranceProgressListener
import androidx.car.app.model.CarText
import androidx.car.app.model.Distance
import androidx.car.app.navigation.model.Maneuver
import androidx.car.app.navigation.model.Step
import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.audio.AudioLink
import java.io.File
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.CountDownLatch
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
import kotlin.math.roundToInt
/**
* Spoken turn-by-turn on the car's directions channel (the GUIDANCE audio sink).
*
* A map app on the phone speaks its own prompts as navigation audio, which Android can't hand to
* Gearslip without root, and which the phone mutes while USB audio runs. So Gearslip reads the
* same turn the instrument cluster gets ([NavTurn]) and says it itself: once when the turn comes
* up, again a kilometre or so out on a long stretch, and once more just before it. The words are
* rendered with the phone's text-to-speech to a file, never played on the phone, and streamed to
* the car, which ducks the music under them the way it does for real Android Auto.
*
* Nothing spoken is logged: road names are where the driver is going.
*/
object DirectionsVoice {
private enum class Stage { FAR, PREPARED, DONE }
private var appContext: Context? = null
private var tts: TextToSpeech? = null
@Volatile private var ttsReady = false
private val pending = ConcurrentHashMap<String, CountDownLatch>()
@Volatile private var link: AudioLink? = null
@Volatile private var running = false
private val speaker = Executors.newSingleThreadExecutor { Thread(it, "gearslip-directions").apply { isDaemon = true } }
@Volatile private var latest = 0
private var turnKey: String? = null
private var stage = Stage.DONE
fun init(context: Context) {
if (tts != null) return
appContext = context.applicationContext
tts = TextToSpeech(appContext) { status -> ttsReady = status == TextToSpeech.SUCCESS }.apply {
setOnUtteranceProgressListener(object : UtteranceProgressListener() {
override fun onStart(utteranceId: String?) {}
override fun onDone(utteranceId: String?) { pending.remove(utteranceId)?.countDown() }
@Deprecated("Deprecated in Java, no replacement on this API level")
override fun onError(utteranceId: String?) { pending.remove(utteranceId)?.countDown() }
})
}
}
/** The car opened its directions sink: start following the map app's turns. */
fun attach(link: AudioLink) {
this.link = link
GearslipLog.i("guidance: the car offered a directions sink at ${link.sampleRate}Hz x${link.channels}")
if (running) return
running = true
turnKey = null
Thread({
while (running) {
runCatching { tick() }.onFailure { GearslipLog.w("guidance: ${it.message}") }
Thread.sleep(TICK_MS)
}
}, "gearslip-directions-watch").apply { isDaemon = true }.start()
}
fun detach(link: AudioLink) {
if (this.link !== link) return
this.link = null
running = false
}
private fun tick() {
if (!CarSettings.speakDirections.value) return
val turn = NavTurn.current()
if (turn == null) {
turnKey = null
return
}
if (turn.rerouting) return
val step = turn.step
val instruction = instruction(step) ?: return
val meters = turn.distance?.let(NavTurn::toMeters)
val close = (turn.seconds != null && turn.seconds <= NEAR_SECONDS) || (meters != null && meters <= NEAR_METERS)
val key = "${step.road.text()}|${step.maneuver?.type}|${step.maneuver?.roundaboutExitNumber}|${step.cue.text()}"
if (key != turnKey) {
turnKey = key
if (step.maneuver?.type == Maneuver.TYPE_DEPART) {
stage = Stage.DONE
return
}
if (close || turn.distance == null) {
say(instruction)
stage = Stage.DONE
} else {
say("In ${spoken(turn.distance)}, $instruction")
stage = if (meters != null && meters > PREPARE_FROM_METERS) Stage.FAR else Stage.PREPARED
}
return
}
when {
stage != Stage.DONE && close -> {
say(instruction)
stage = Stage.DONE
}
stage == Stage.FAR && meters != null && meters <= PREPARE_AT_METERS -> {
say("In ${spoken(turn.distance)}, $instruction")
stage = Stage.PREPARED
}
}
}
/** What to do, in words: built from the manoeuvre when the app names one, else the app's own text. */
private fun instruction(step: Step): String? {
val road = step.road.text()
val onto = if (road.isBlank()) "" else " onto $road"
val toward = if (road.isBlank()) "" else " toward $road"
val maneuver = step.maneuver
val built = when (maneuver?.type) {
Maneuver.TYPE_TURN_SLIGHT_LEFT -> "Turn slightly left$onto"
Maneuver.TYPE_TURN_SLIGHT_RIGHT -> "Turn slightly right$onto"
Maneuver.TYPE_TURN_NORMAL_LEFT -> "Turn left$onto"
Maneuver.TYPE_TURN_NORMAL_RIGHT -> "Turn right$onto"
Maneuver.TYPE_TURN_SHARP_LEFT -> "Turn sharply left$onto"
Maneuver.TYPE_TURN_SHARP_RIGHT -> "Turn sharply right$onto"
Maneuver.TYPE_U_TURN_LEFT, Maneuver.TYPE_U_TURN_RIGHT -> "Make a U-turn"
Maneuver.TYPE_KEEP_LEFT -> "Keep left$toward"
Maneuver.TYPE_KEEP_RIGHT -> "Keep right$toward"
Maneuver.TYPE_FORK_LEFT -> "Keep left at the fork$toward"
Maneuver.TYPE_FORK_RIGHT -> "Keep right at the fork$toward"
Maneuver.TYPE_ON_RAMP_SLIGHT_LEFT, Maneuver.TYPE_ON_RAMP_NORMAL_LEFT, Maneuver.TYPE_ON_RAMP_SHARP_LEFT,
Maneuver.TYPE_ON_RAMP_U_TURN_LEFT -> "Take the ramp on the left$onto"
Maneuver.TYPE_ON_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_ON_RAMP_NORMAL_RIGHT, Maneuver.TYPE_ON_RAMP_SHARP_RIGHT,
Maneuver.TYPE_ON_RAMP_U_TURN_RIGHT -> "Take the ramp on the right$onto"
Maneuver.TYPE_OFF_RAMP_SLIGHT_LEFT, Maneuver.TYPE_OFF_RAMP_NORMAL_LEFT -> "Take the exit on the left$toward"
Maneuver.TYPE_OFF_RAMP_SLIGHT_RIGHT, Maneuver.TYPE_OFF_RAMP_NORMAL_RIGHT -> "Take the exit on the right$toward"
Maneuver.TYPE_MERGE_LEFT -> "Merge left$onto"
Maneuver.TYPE_MERGE_RIGHT -> "Merge right$onto"
Maneuver.TYPE_MERGE_SIDE_UNSPECIFIED -> "Merge$onto"
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CW_WITH_ANGLE,
Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW, Maneuver.TYPE_ROUNDABOUT_ENTER_AND_EXIT_CCW_WITH_ANGLE ->
maneuver.roundaboutExitNumber.takeIf { it in 1..ORDINALS.size }
?.let { "At the roundabout, take the ${ORDINALS[it - 1]} exit$onto" }
?: "Go through the roundabout$onto"
Maneuver.TYPE_ROUNDABOUT_ENTER_CW, Maneuver.TYPE_ROUNDABOUT_ENTER_CCW -> "Enter the roundabout"
Maneuver.TYPE_ROUNDABOUT_EXIT_CW, Maneuver.TYPE_ROUNDABOUT_EXIT_CCW -> "Exit the roundabout$onto"
Maneuver.TYPE_STRAIGHT -> "Continue straight$onto"
Maneuver.TYPE_NAME_CHANGE -> "Continue$onto"
Maneuver.TYPE_FERRY_BOAT, Maneuver.TYPE_FERRY_BOAT_LEFT, Maneuver.TYPE_FERRY_BOAT_RIGHT -> "Take the ferry"
Maneuver.TYPE_FERRY_TRAIN, Maneuver.TYPE_FERRY_TRAIN_LEFT, Maneuver.TYPE_FERRY_TRAIN_RIGHT -> "Take the train"
Maneuver.TYPE_DESTINATION, Maneuver.TYPE_DESTINATION_STRAIGHT -> "Your destination is ahead"
Maneuver.TYPE_DESTINATION_LEFT -> "Your destination is on the left"
Maneuver.TYPE_DESTINATION_RIGHT -> "Your destination is on the right"
else -> null
}
return built ?: step.cue.text().ifBlank { null }
}
/** The app's own rounded distance, read the way a person would say it. */
private fun spoken(d: Distance): String {
val n = d.displayDistance
val number = if (n % 1.0 == 0.0) n.toInt().toString() else "%.1f".format(n)
val one = n == 1.0
return when (d.displayUnit) {
Distance.UNIT_KILOMETERS, Distance.UNIT_KILOMETERS_P1 -> if (one) "1 kilometer" else "$number kilometers"
Distance.UNIT_MILES, Distance.UNIT_MILES_P1 -> when (n) {
0.25 -> "a quarter mile"
0.5 -> "half a mile"
0.75 -> "three quarters of a mile"
1.0 -> "1 mile"
else -> "$number miles"
}
Distance.UNIT_FEET -> "$number feet"
Distance.UNIT_YARDS -> "$number yards"
else -> "${n.roundToInt()} meters"
}
}
/** Newest wins: a prompt still waiting when the next one arrives is already out of date. */
private fun say(text: String) {
val id = ++latest
speaker.execute {
if (id != latest) return@execute
runCatching { play(text, id) }.onFailure { GearslipLog.w("guidance: could not speak a turn: ${it.message}") }
}
}
private fun play(text: String, id: Int) {
val engine = tts ?: return
val context = appContext ?: return
val sink = link ?: return
if (!ttsReady) {
GearslipLog.w("guidance: text-to-speech isn't ready, a turn went unspoken")
return
}
val file = File(context.cacheDir, "directions.wav")
val utterance = "gearslip-directions-$id"
val done = CountDownLatch(1)
pending[utterance] = done
if (engine.synthesizeToFile(text, Bundle(), file, utterance) != TextToSpeech.SUCCESS) {
pending.remove(utterance)
return
}
if (!done.await(SYNTH_TIMEOUT_S, TimeUnit.SECONDS)) {
pending.remove(utterance)
GearslipLog.w("guidance: text-to-speech took too long, a turn went unspoken")
return
}
val pcm = Wav.read(file)?.let { Wav.convert(it, sink.sampleRate, sink.channels) } ?: run {
GearslipLog.w("guidance: text-to-speech gave audio Gearslip can't read")
return
}
stream(sink, pcm)
GearslipLog.i("guidance: spoke a turn (${pcm.size / (2 * sink.channels) * 1000 / sink.sampleRate} ms)")
}
/** Start, the words in 20 ms chunks at about real time, a little silence, Stop. */
private fun stream(sink: AudioLink, pcm: ByteArray) {
val frame = 2 * sink.channels
val chunk = sink.sampleRate / 50 * frame
val chunkMs = 20L
val padded = pcm + ByteArray(sink.sampleRate / 5 * frame) // 200 ms tail so the last word isn't clipped
sink.start()
try {
val began = System.currentTimeMillis()
var offset = 0
var sent = 0
while (offset < padded.size && link === sink) {
val size = minOf(chunk, padded.size - offset)
sink.write(padded.copyOfRange(offset, offset + size), size)
offset += size
sent++
val ahead = sent * chunkMs - (System.currentTimeMillis() - began)
if (ahead > LEAD_MS) Thread.sleep(ahead - LEAD_MS)
}
// Let the car play what it has before closing the stream on it.
Thread.sleep(LEAD_MS)
} finally {
sink.stop()
}
}
private fun CarText?.text() = this?.toString().orEmpty()
/** 16-bit PCM from the WAV text-to-speech writes, reshaped for the car's sink. */
private object Wav {
class Pcm(val samples: ShortArray, val rate: Int, val channels: Int)
fun read(file: File): Pcm? {
val bytes = file.readBytes()
if (bytes.size < 12 || String(bytes, 0, 4) != "RIFF" || String(bytes, 8, 4) != "WAVE") return null
val buf = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN)
var pos = 12
var rate = 0
var channels = 0
var bits = 0
while (pos + 8 <= bytes.size) {
val name = String(bytes, pos, 4)
val size = buf.getInt(pos + 4)
val body = pos + 8
// Some engines write a streaming WAV with a zero or oversized data length.
val end = if (size <= 0 || body + size > bytes.size) bytes.size else body + size
when (name) {
"fmt " -> {
channels = buf.getShort(body + 2).toInt()
rate = buf.getInt(body + 4)
bits = buf.getShort(body + 14).toInt()
}
"data" -> {
if (bits != 16 || rate <= 0 || channels <= 0) return null
val samples = ShortArray((end - body) / 2)
buf.position(body)
buf.asShortBuffer().get(samples, 0, samples.size)
return Pcm(samples, rate, channels)
}
}
pos = end + (end and 1)
}
return null
}
/** Mixes to mono, resamples linearly, then copies to each of the car's channels. */
fun convert(pcm: Pcm, rate: Int, channels: Int): ByteArray {
val frames = pcm.samples.size / pcm.channels
val mono = FloatArray(frames) { f ->
var sum = 0f
for (c in 0 until pcm.channels) sum += pcm.samples[f * pcm.channels + c]
sum / pcm.channels
}
val outFrames = (frames.toLong() * rate / pcm.rate).toInt()
val out = ByteBuffer.allocate(outFrames * channels * 2).order(ByteOrder.LITTLE_ENDIAN)
val step = pcm.rate.toDouble() / rate
for (i in 0 until outFrames) {
val at = i * step
val j = at.toInt().coerceAtMost(frames - 1)
val next = (j + 1).coerceAtMost(frames - 1)
val v = mono[j] + (mono[next] - mono[j]) * (at - j).toFloat()
val s = v.roundToInt().coerceIn(Short.MIN_VALUE.toInt(), Short.MAX_VALUE.toInt()).toShort()
repeat(channels) { out.putShort(s) }
}
return out.array()
}
}
private const val TICK_MS = 500L
private const val NEAR_SECONDS = 12L
private const val NEAR_METERS = 120
private const val PREPARE_FROM_METERS = 2_500
private const val PREPARE_AT_METERS = 1_000
private const val LEAD_MS = 100L
private const val SYNTH_TIMEOUT_S = 10L
private val ORDINALS = listOf("first", "second", "third", "fourth", "fifth", "sixth", "seventh", "eighth")
}
@@ -0,0 +1,84 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.IndicationNodeFactory
import androidx.compose.foundation.interaction.FocusInteraction
import androidx.compose.foundation.interaction.InteractionSource
import androidx.compose.material3.ripple
import androidx.compose.ui.geometry.CornerRadius
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.geometry.Size
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.drawscope.ContentDrawScope
import androidx.compose.ui.graphics.drawscope.Stroke
import androidx.compose.ui.node.DelegatableNode
import androidx.compose.ui.node.DelegatingNode
import androidx.compose.ui.node.DrawModifierNode
import androidx.compose.ui.node.invalidateDraw
import androidx.compose.ui.unit.dp
import kotlinx.coroutines.launch
/**
* What every car button shows: the usual ripple when pressed, and a ring in the accent color
* while the knob or D-pad is on it. Cars without a knob never see the ring, since nothing takes
* focus while the screen is in touch mode.
*
* It replaces the theme's ripple (LocalIndication) for the whole car UI, so any `clickable` that
* doesn't pass its own indication gets the ring for free.
*/
internal class FocusRingIndication(private val color: Color) : IndicationNodeFactory {
private val ripple = ripple()
override fun create(interactionSource: InteractionSource): DelegatableNode =
FocusRingNode(ripple.create(interactionSource), interactionSource, color)
override fun equals(other: Any?) = other is FocusRingIndication && other.color == color
override fun hashCode() = color.hashCode()
}
private class FocusRingNode(
rippleNode: DelegatableNode,
private val source: InteractionSource,
private val color: Color,
) : DelegatingNode(), DrawModifierNode {
private var focused = false
init { delegate(rippleNode) }
override fun onAttach() {
coroutineScope.launch {
var holding = 0
source.interactions.collect {
when (it) {
is FocusInteraction.Focus -> holding++
is FocusInteraction.Unfocus -> holding--
}
val now = holding > 0
if (now != focused) {
focused = now
invalidateDraw()
}
}
}
}
override fun ContentDrawScope.draw() {
drawContent()
if (!focused) return
// Inside the button's bounds, since most buttons clip to their shape: drawn outside, a
// rounded tile would cut its own ring off.
val width = RING_WIDTH.toPx()
val radius = minOf(CORNER.toPx(), size.minDimension / 2)
drawRoundRect(
color,
topLeft = Offset(width / 2, width / 2),
size = Size(size.width - width, size.height - width),
cornerRadius = CornerRadius(radius),
style = Stroke(width),
)
}
private companion object {
val RING_WIDTH = 3.dp
val CORNER = 16.dp
}
}
@@ -0,0 +1,93 @@
package app.seb3thehacker.gearslip.car
import android.content.BroadcastReceiver
import android.content.Context
import android.content.Intent
import android.content.IntentFilter
import android.location.LocationManager
import android.os.Handler
import android.os.Looper
import app.seb3thehacker.gearslip.GearslipLog
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
/**
* Map apps can't find the car with the phone's location turned off. Ten seconds after the car
* shows Gearslip, this checks, and the car says so until location comes on or the drive ends.
*/
object LocationCheck {
private val _off = MutableStateFlow(false)
/** True while the phone's location is turned off during a drive. */
val off: StateFlow<Boolean> = _off
private val _carDismissed = MutableStateFlow(false)
/** The driver tapped OK on the car's card; it stays away for the rest of this drive. */
val carDismissed: StateFlow<Boolean> = _carDismissed
private val log = GearslipLog.tagged("location")
private val main = Handler(Looper.getMainLooper())
private var appContext: Context? = null
private var armed = false
private var receiver: BroadcastReceiver? = null
fun init(context: Context) {
appContext = context.applicationContext
}
/** The car shows Gearslip. Called again whenever the video restarts, so it arms once per drive. */
fun onProjecting() = main.post {
if (armed) return@post
armed = true
main.postDelayed(::start, WAIT_MS)
}
fun onCarDisconnected() = main.post {
armed = false
main.removeCallbacksAndMessages(null)
stopWatching()
_off.value = false
_carDismissed.value = false
}
fun dismissOnCar() {
_carDismissed.value = true
}
/** "Don't show again" on the car: turns the warning off in Settings, for every car. */
fun neverAgain() {
CarSettings.setLocationWarning(false)
log.i("location warning turned off")
_carDismissed.value = true
_off.value = false
}
private fun start() {
if (!armed) return
check()
// Turning location on clears the warning, and turning it off again brings it back.
val context = appContext ?: return
val r = object : BroadcastReceiver() {
override fun onReceive(context: Context, intent: Intent) = check()
}
context.registerReceiver(r, IntentFilter(LocationManager.MODE_CHANGED_ACTION), Context.RECEIVER_EXPORTED)
receiver = r
}
private fun check() {
if (!armed) return
val manager = appContext?.getSystemService(LocationManager::class.java) ?: return
val off = CarSettings.locationWarning.value && !manager.isLocationEnabled
if (_off.value == off) return
_off.value = off
log.i(if (off) "the phone's location is off" else "the phone's location is on")
}
private fun stopWatching() {
val r = receiver ?: return
receiver = null
runCatching { appContext?.unregisterReceiver(r) }
}
private const val WAIT_MS = 10_000L
}
@@ -30,12 +30,14 @@ import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight import androidx.compose.material.icons.automirrored.filled.KeyboardArrowRight
import androidx.compose.material.icons.filled.ArrowBack import androidx.compose.material.icons.filled.ArrowBack
import androidx.compose.material.icons.filled.PlayArrow import androidx.compose.material.icons.filled.PlayArrow
import androidx.compose.material.icons.filled.Search
import androidx.compose.material3.Icon import androidx.compose.material3.Icon
import androidx.compose.material3.MaterialTheme import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface import androidx.compose.material3.Surface
import androidx.compose.material3.Text import androidx.compose.material3.Text
import androidx.compose.material3.TextButton import androidx.compose.material3.TextButton
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
@@ -59,7 +61,6 @@ import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextOverflow import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.compose.collectAsStateWithLifecycle
import app.seb3thehacker.gearslip.audio.CarAudio
import app.seb3thehacker.gearslip.media.CarMedia import app.seb3thehacker.gearslip.media.CarMedia
import app.seb3thehacker.gearslip.media.MediaApp import app.seb3thehacker.gearslip.media.MediaApp
import app.seb3thehacker.gearslip.media.MediaArt import app.seb3thehacker.gearslip.media.MediaArt
@@ -69,6 +70,9 @@ import app.seb3thehacker.gearslip.media.MediaEntry
import app.seb3thehacker.gearslip.media.NowPlaying import app.seb3thehacker.gearslip.media.NowPlaying
import app.seb3thehacker.gearslip.media.QueueTrack import app.seb3thehacker.gearslip.media.QueueTrack
import android.media.session.PlaybackState import android.media.session.PlaybackState
import kotlinx.coroutines.delay
import androidx.compose.foundation.clickable
import androidx.compose.foundation.layout.PaddingValues
/** /**
* A media app on the car screen: what it is playing and its controls on the left, the tree it * A media app on the car screen: what it is playing and its controls on the left, the tree it
@@ -83,8 +87,9 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
val queue by media.queue.collectAsStateWithLifecycle() val queue by media.queue.collectAsStateWithLifecycle()
val browse by media.browse.collectAsStateWithLifecycle() val browse by media.browse.collectAsStateWithLifecycle()
val rejection by media.rejection.collectAsStateWithLifecycle() val rejection by media.rejection.collectAsStateWithLifecycle()
val capture by CarAudio.capture.collectAsStateWithLifecycle() val search by media.search.collectAsStateWithLifecycle()
var tab by remember { mutableStateOf(Tab.NOW_PLAYING) } var tab by remember { mutableStateOf(Tab.NOW_PLAYING) }
var searching by remember { mutableStateOf(false) }
// The connection outlives this screen: the home screen's player is the same one. // The connection outlives this screen: the home screen's player is the same one.
LaunchedEffect(app) { CarServices.openMedia(app) } LaunchedEffect(app) { CarServices.openMedia(app) }
@@ -111,6 +116,12 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
} }
}, },
) )
CarMedia.Rejection.NEEDS_APP_RUNNING -> Notice(
"${app.label} isn't running.",
"${app.label} plays through Gearslip's own player, but only while it's " +
"open on the phone. Open it once and it keeps playing here in the car.",
action = "Open ${app.label}" to { media.launchAndConnect(app) },
)
CarMedia.Rejection.REFUSED -> Notice( CarMedia.Rejection.REFUSED -> Notice(
"${app.label} would not let Gearslip browse it.", "${app.label} would not let Gearslip browse it.",
"Some media apps only accept Google's own host.", "Some media apps only accept Google's own host.",
@@ -122,6 +133,29 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
return@Surface return@Surface
} }
if (searching) {
SearchPanel(
appLabel = app.label,
media = media,
canPickForYou = now.canSearch,
onPlayed = {
searching = false
tab = Tab.NOW_PLAYING
},
onOpened = {
searching = false
tab = Tab.BROWSE
},
onPickForYou = { query ->
media.search(query)
searching = false
tab = Tab.NOW_PLAYING
},
onClose = { searching = false },
)
return@Surface
}
// Two columns, split the same way on every tab so nothing jumps when switching: the app // Two columns, split the same way on every tab so nothing jumps when switching: the app
// and the art (or, beside a list, the compact player) on the left; the tabs across the // and the art (or, beside a list, the compact player) on the left; the tabs across the
// top of the right column, over whatever the tab shows. // top of the right column, over whatever the tab shows.
@@ -137,7 +171,6 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
} else { } else {
CompactPlayer(now, media, Modifier.weight(1f).fillMaxWidth()) CompactPlayer(now, media, Modifier.weight(1f).fillMaxWidth())
} }
AudioBadge(capture)
} }
Column(Modifier.weight(1.2f).fillMaxHeight()) { Column(Modifier.weight(1.2f).fillMaxHeight()) {
Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) { Row(Modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
@@ -145,7 +178,18 @@ fun MediaScreen(app: MediaApp, onExit: () -> Unit) {
MediaTab("Browse", tab == Tab.BROWSE, Modifier.weight(1f)) { tab = Tab.BROWSE } MediaTab("Browse", tab == Tab.BROWSE, Modifier.weight(1f)) { tab = Tab.BROWSE }
MediaTab("Up next", tab == Tab.QUEUE, Modifier.weight(1f)) { tab = Tab.QUEUE } MediaTab("Up next", tab == Tab.QUEUE, Modifier.weight(1f)) { tab = Tab.QUEUE }
MediaTab("Lyrics", tab == Tab.LYRICS, Modifier.weight(1f)) { tab = Tab.LYRICS } MediaTab("Lyrics", tab == Tab.LYRICS, Modifier.weight(1f)) { tab = Tab.LYRICS }
if (now.canSearch || media.canSearchLibrary) {
GsIconButton(
Icons.Filled.Search,
"Search ${app.label}",
{ searching = true },
colors = GsColors(MaterialTheme.colorScheme.surfaceContainerHigh, MaterialTheme.colorScheme.onSurface),
size = 44.dp,
shape = MaterialTheme.shapes.large,
)
}
} }
search?.let { SearchStatus(it, app.label, media::dismissSearch) }
Spacer(Modifier.height(12.dp)) Spacer(Modifier.height(12.dp))
Box(Modifier.weight(1f).fillMaxWidth()) { Box(Modifier.weight(1f).fillMaxWidth()) {
when (tab) { when (tab) {
@@ -186,6 +230,135 @@ private fun AppIdentity(app: MediaApp, onExit: () -> Unit, modifier: Modifier =
private enum class Tab { NOW_PLAYING, BROWSE, QUEUE, LYRICS } private enum class Tab { NOW_PLAYING, BROWSE, QUEUE, LYRICS }
// --- search ----------------------------------------------------------------------------------
/**
* Types a search for the app to play. The app picks the result and plays it, the way it would
* for "play X" from a voice assistant, so there's no list to choose from.
*/
/**
* Search inside the media app. It asks the app's library for matches half a second after the
* driver stops typing and lists them, so the driver picks exactly what plays: a song plays, an
* album or artist opens in Browse. The search key hides the keyboard to show more results; the
* field brings it back. An app with no library search can still be told to play its own best
* match, which is all the panel could do before.
*/
@Composable
private fun SearchPanel(
appLabel: String,
media: CarMedia,
canPickForYou: Boolean,
onPlayed: () -> Unit,
onOpened: () -> Unit,
onPickForYou: (String) -> Unit,
onClose: () -> Unit,
) {
var query by remember { mutableStateOf("") }
var typing by remember { mutableStateOf(true) }
val results by media.results.collectAsStateWithLifecycle()
LaunchedEffect(query) {
delay(SEARCH_PAUSE_MS)
media.searchLibrary(query)
}
DisposableEffect(Unit) { onDispose { media.clearResults() } }
val trimmed = query.trim()
val current = results?.takeIf { it.query == trimmed }
val unsupported = current?.supported == false
Column(Modifier.fillMaxSize()) {
Row(
Modifier.fillMaxWidth().padding(horizontal = 16.dp, vertical = 10.dp),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(8.dp),
) {
GsIconButton(Icons.Filled.ArrowBack, "Back", onClose, size = ICON_BUTTON_SIZE)
SearchField(
query,
"Search $appLabel",
Modifier.weight(1f).clickable { typing = true },
onClear = {
query = ""
typing = true
},
)
}
Box(Modifier.weight(1f).fillMaxWidth().padding(horizontal = 16.dp)) {
val entries = current?.entries
when {
trimmed.isEmpty() -> SearchNote("Type a song, artist, album, or playlist.")
unsupported && canPickForYou ->
SearchNote("$appLabel can't list results. Press search, and $appLabel picks what to play.")
unsupported -> SearchNote("$appLabel can't search from the car.")
entries == null -> SearchNote("Searching $appLabel…")
entries.isEmpty() -> SearchNote("$appLabel found nothing for \"$trimmed\".")
else -> LazyColumn(verticalArrangement = Arrangement.spacedBy(8.dp), contentPadding = PaddingValues(bottom = 8.dp)) {
items(entries) { entry ->
EntryRow(entry, media) {
if (entry.browsable) {
media.select(entry)
onOpened()
} else {
media.playAll(entry)
onPlayed()
}
}
}
}
}
}
if (typing) {
CarKeyboard(
text = query,
onTextChange = { query = it },
onSubmit = {
when {
trimmed.isEmpty() -> Unit
unsupported && canPickForYou -> onPickForYou(trimmed)
else -> typing = false
}
},
onDismiss = { if (trimmed.isEmpty()) onClose() else typing = false },
)
}
}
}
@Composable
private fun SearchNote(text: String) {
Box(Modifier.fillMaxSize(), contentAlignment = Alignment.Center) {
Text(
text,
style = MaterialTheme.typography.bodyLarge,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = androidx.compose.ui.text.style.TextAlign.Center,
)
}
}
/** How long typing has to pause before the library is searched. */
private const val SEARCH_PAUSE_MS = 1_000L
/** A line under the tabs while the app works on a search, and if it plays nothing for it. */
@Composable
private fun SearchStatus(search: CarMedia.Search, appLabel: String, onDismiss: () -> Unit) {
Surface(
onClick = onDismiss,
shape = MaterialTheme.shapes.large,
color = MaterialTheme.colorScheme.surfaceContainerHigh,
modifier = Modifier.fillMaxWidth().padding(top = 8.dp),
) {
Text(
if (search.ignored) "$appLabel didn't play anything for \"${search.query}\"."
else "Searching $appLabel for \"${search.query}\"…",
style = MaterialTheme.typography.bodyLarge,
maxLines = 2,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.padding(horizontal = 16.dp, vertical = 10.dp),
)
}
}
/** A bigger, easier-to-hit tab than a plain [TextButton] - this row is the only navigation on the screen. */ /** A bigger, easier-to-hit tab than a plain [TextButton] - this row is the only navigation on the screen. */
@Composable @Composable
private fun MediaTab(label: String, selected: Boolean, modifier: Modifier, onClick: () -> Unit) { private fun MediaTab(label: String, selected: Boolean, modifier: Modifier, onClick: () -> Unit) {
@@ -270,17 +443,6 @@ private sealed interface LyricsState {
class Found(val result: LyricsResult) : LyricsState class Found(val result: LyricsResult) : LyricsState
} }
@Composable
private fun AudioBadge(capture: CarAudio.Capture) {
val text = when (capture) {
CarAudio.Capture.ON -> "Audio to car"
CarAudio.Capture.ASKING -> "Allow audio capture on the phone"
CarAudio.Capture.DENIED -> "Audio stays on the phone"
CarAudio.Capture.OFF -> return
}
Text(text, style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
@Composable @Composable
private fun Notice(title: String, detail: String?, action: Pair<String, () -> Unit>? = null) { private fun Notice(title: String, detail: String?, action: Pair<String, () -> Unit>? = null) {
Column( Column(
@@ -484,13 +646,13 @@ private fun QueueRow(track: QueueTrack, media: CarMedia) {
} }
@Composable @Composable
private fun EntryRow(entry: MediaEntry, media: CarMedia) { private fun EntryRow(entry: MediaEntry, media: CarMedia, onClick: () -> Unit = { media.select(entry) }) {
val context = LocalContext.current val context = LocalContext.current
val icon by produceState(entry.iconBitmap, entry.iconUri) { val icon by produceState(entry.iconBitmap, entry.iconUri) {
value = entry.iconBitmap ?: MediaArt.load(context, entry.iconUri) value = entry.iconBitmap ?: MediaArt.load(context, entry.iconUri)
} }
Surface( Surface(
onClick = { media.select(entry) }, onClick = onClick,
shape = MaterialTheme.shapes.large, shape = MaterialTheme.shapes.large,
color = MaterialTheme.colorScheme.surfaceVariant, color = MaterialTheme.colorScheme.surfaceVariant,
modifier = Modifier.fillMaxWidth(), modifier = Modifier.fillMaxWidth(),
@@ -40,27 +40,48 @@ import app.seb3thehacker.gearslip.car.theme.GsButton
import app.seb3thehacker.gearslip.car.theme.GsTone import app.seb3thehacker.gearslip.car.theme.GsTone
import app.seb3thehacker.gearslip.host.CarAppConnection import app.seb3thehacker.gearslip.host.CarAppConnection
import app.seb3thehacker.gearslip.media.CarMedia import app.seb3thehacker.gearslip.media.CarMedia
import app.seb3thehacker.gearslip.CarFocus
import androidx.compose.ui.graphics.SolidColor
import androidx.compose.ui.graphics.vector.PathParser
/** What a built-in app's id carries in a pin, so it can't collide with an installed app's component. */ /** What a built-in app's id carries in a pin, so it can't collide with an installed app's component. */
internal const val BUILT_IN_PREFIX = "gearslip:" internal const val BUILT_IN_PREFIX = "gearslip:"
/** One of Gearslip's own apps: a screen, with nothing running behind it once you leave. */ /** Material's "directions car", drawn here because only the core icon set is bundled. */
private val CarGlyph: ImageVector = ImageVector.Builder("Car", 24.dp, 24.dp, 24f, 24f).addPath(
PathParser().parsePathString(
"M18.92,6.01C18.72,5.42 18.16,5 17.5,5h-11c-0.66,0 -1.21,0.42 -1.42,1.01L3,12v8c0,0.55 0.45,1 1,1h1" +
"c0.55,0 1,-0.45 1,-1v-1h12v1c0,0.55 0.45,1 1,1h1c0.55,0 1,-0.45 1,-1v-8l-2.08,-5.99z" +
"M6.5,16c-0.83,0 -1.5,-0.67 -1.5,-1.5S5.67,13 6.5,13s1.5,0.67 1.5,1.5S7.33,16 6.5,16z" +
"M17.5,16c-0.83,0 -1.5,-0.67 -1.5,-1.5s0.67,-1.5 1.5,-1.5 1.5,0.67 1.5,1.5 -0.67,1.5 -1.5,1.5z" +
"M5,11l1.5,-4.5h11L19,11L5,11z",
).toNodes(),
fill = SolidColor(Color.Black),
).build()
/**
* One of Gearslip's own apps: a screen, with nothing running behind it once you leave. [screen] is
* null for an action like Exit, which has no screen and so never shows as open.
*/
internal class BuiltInApp( internal class BuiltInApp(
val id: String, val id: String,
val label: String, val label: String,
val glyph: ImageVector, val glyph: ImageVector,
val screen: CarScreen, val screen: CarScreen?,
val open: (CarNavigator) -> Unit, val open: (CarNavigator) -> Unit,
) )
internal object BuiltInApps { internal object BuiltInApps {
val Web = BuiltInApp("web", "Web", Icons.Filled.Search, CarScreen.App("web")) { it.open("web") } val Web = BuiltInApp("web", "Browser", Icons.Filled.Search, CarScreen.App("web")) { it.open("web") }
val ScreenSharing = BuiltInApp("phone", "Screen sharing", Icons.Filled.Share, CarScreen.App("phone")) { it.open("phone") } val ScreenSharing = BuiltInApp("phone", "Screen sharing", Icons.Filled.Share, CarScreen.App("phone")) { it.open("phone") }
val Phone = BuiltInApp("dialer", "Phone", Icons.Filled.Call, CarScreen.App("dialer")) { it.open("dialer") } val Phone = BuiltInApp("dialer", "Phone", Icons.Filled.Call, CarScreen.App("dialer")) { it.open("dialer") }
val VehicleData = BuiltInApp("vehicle", "Vehicle data", Icons.Filled.Info, CarScreen.VehicleData) { it.vehicleData() } val VehicleData = BuiltInApp("vehicle", "Vehicle data", Icons.Filled.Info, CarScreen.VehicleData) { it.vehicleData() }
val Settings = BuiltInApp("settings", "Settings", Icons.Filled.Settings, CarScreen.Settings) { it.settings() } val Settings = BuiltInApp("settings", "Settings", Icons.Filled.Settings, CarScreen.Settings) { it.settings() }
val all = listOf(Web, ScreenSharing, Phone, VehicleData, Settings) /** Hands the screen back to the car's own interface, like Android Auto's Exit. */
val Exit = BuiltInApp("exit", "Exit", CarGlyph, screen = null) { CarFocus.exitToCar() }
val all = listOf(Web, ScreenSharing, Phone, VehicleData, Settings, Exit)
val entries: List<Entry> = all.map { Entry(it.label, icon = null, builtIn = it) } val entries: List<Entry> = all.map { Entry(it.label, icon = null, builtIn = it) }
} }
@@ -110,7 +131,7 @@ internal fun rememberRunningApps(): Map<String, RunningSlot> {
else -> false else -> false
} }
mediaApp?.component?.flattenToString()?.takeIf { mediaLive }?.let { put(it, RunningSlot.MEDIA) } mediaApp?.component?.flattenToString()?.takeIf { mediaLive }?.let { put(it, RunningSlot.MEDIA) }
BuiltInApps.all.firstOrNull { it.screen == current }?.let { put(BUILT_IN_PREFIX + it.id, RunningSlot.SCREEN) } BuiltInApps.all.firstOrNull { it.screen != null && it.screen == current }?.let { put(BUILT_IN_PREFIX + it.id, RunningSlot.SCREEN) }
(current as? CarScreen.Messages)?.let { put(MESSAGING_PREFIX + it.packageName, RunningSlot.SCREEN) } (current as? CarScreen.Messages)?.let { put(MESSAGING_PREFIX + it.packageName, RunningSlot.SCREEN) }
} }
} }
@@ -148,8 +169,8 @@ internal fun closeApp(slot: RunningSlot, navigator: CarNavigator) {
} }
/** /**
* What a long press on an app offers - pin it to or unpin it from the nav bar, and close it if * What a long press on an app offers - pin it to or unpin it from the nav bar, move it along the
* it's open - as a card in the middle of the screen with car-sized buttons, over a scrim that * launcher, and close it if it's open - as a card in the middle of the screen with car-sized buttons, over a scrim that
* dismisses it. Drawn in the car UI's own layout rather than a system popup window, which the * dismisses it. Drawn in the car UI's own layout rather than a system popup window, which the
* car's virtual display has no reason to host well. * car's virtual display has no reason to host well.
*/ */
@@ -218,6 +239,7 @@ internal fun AppMenu(navigator: CarNavigator, running: Map<String, RunningSlot>)
style = MaterialTheme.typography.titleMedium, style = MaterialTheme.typography.titleMedium,
) )
} }
if (navigator.current == CarScreen.Apps) MoveButtons(id)
if (slot != null) { if (slot != null) {
GsButton( GsButton(
onClick = { onClick = {
@@ -237,3 +259,41 @@ internal fun AppMenu(navigator: CarNavigator, running: Map<String, RunningSlot>)
} }
} }
} }
/**
* Moves the app one place left or right on the launcher. The menu stays open so a tile can be
* walked several places, and the grid behind the scrim shows where it went.
*/
@Composable
private fun MoveButtons(id: String) {
val custom by CarSettings.appOrder.collectAsState()
val order = launcherOrder(rememberAllEntries(), rememberVehicleDataShown(), custom).mapNotNull { it.componentId }
val index = order.indexOf(id)
if (index < 0) return
fun move(by: Int) {
val next = order.toMutableList()
java.util.Collections.swap(next, index, index + by)
CarSettings.setAppOrder(next)
}
Row(horizontalArrangement = Arrangement.spacedBy(12.dp)) {
GsButton(
onClick = { move(-1) },
tone = GsTone.Tonal,
enabled = index > 0,
modifier = Modifier.weight(1f).height(56.dp),
) { Text("Move left", style = MaterialTheme.typography.titleMedium) }
GsButton(
onClick = { move(1) },
tone = GsTone.Tonal,
enabled = index < order.lastIndex,
modifier = Modifier.weight(1f).height(56.dp),
) { Text("Move right", style = MaterialTheme.typography.titleMedium) }
}
if (custom.isNotEmpty()) {
GsButton(
onClick = { CarSettings.setAppOrder(emptyList()) },
tone = GsTone.Neutral,
modifier = Modifier.fillMaxWidth().height(56.dp),
) { Text("Reset app order", style = MaterialTheme.typography.titleMedium) }
}
}
@@ -53,6 +53,8 @@ fun PhoneAppsScreen() {
} }
Column(Modifier.fillMaxSize().padding(horizontal = 20.dp, vertical = 12.dp)) { Column(Modifier.fillMaxSize().padding(horizontal = 20.dp, vertical = 12.dp)) {
ScreenHeader("Screen sharing")
Spacer(Modifier.height(12.dp))
if (!TouchRelayService.enabled) { if (!TouchRelayService.enabled) {
TouchRelayWarning() TouchRelayWarning()
Spacer(Modifier.height(12.dp)) Spacer(Modifier.height(12.dp))
@@ -32,6 +32,7 @@ import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface import androidx.compose.material3.Surface
import androidx.compose.material3.Text import androidx.compose.material3.Text
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.produceState import androidx.compose.runtime.produceState
@@ -89,6 +90,31 @@ internal fun placeCall(context: Context, number: String) {
} }
} }
/** A call another car screen wants to place - a number on a place page - waiting on the driver. */
internal object CallPrompt {
val pending = kotlinx.coroutines.flow.MutableStateFlow<Contact?>(null)
fun ask(name: String, number: String) {
pending.value = Contact(name, number)
}
}
/** The dialer's own confirmation, drawn over the whole car screen for [CallPrompt]. */
@Composable
internal fun CallPromptOverlay() {
val context = LocalContext.current
val contact by CallPrompt.pending.collectAsState()
val c = contact ?: return
ConfirmCallOverlay(
c,
onConfirm = {
CallPrompt.pending.value = null
placeCall(context, c.number)
},
onDismiss = { CallPrompt.pending.value = null },
)
}
private const val TAB_CONTACTS = 0 private const val TAB_CONTACTS = 0
private const val TAB_KEYPAD = 1 private const val TAB_KEYPAD = 1
@@ -101,13 +127,7 @@ fun PhoneDialerScreen() {
Column(Modifier.fillMaxSize().padding(16.dp), verticalArrangement = Arrangement.spacedBy(16.dp)) { Column(Modifier.fillMaxSize().padding(16.dp), verticalArrangement = Arrangement.spacedBy(16.dp)) {
// The tabs share the title's row, so the list and keypad get the height back. // The tabs share the title's row, so the list and keypad get the height back.
Row(Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) { ScreenHeader("Phone") {
Text(
"Phone",
style = MaterialTheme.typography.headlineSmall,
fontWeight = FontWeight.Bold,
modifier = Modifier.weight(1f),
)
if (hasPermissions) { if (hasPermissions) {
Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) { Row(horizontalArrangement = Arrangement.spacedBy(8.dp)) {
DialerTab("Contacts", tab == TAB_CONTACTS, Modifier.width(140.dp)) { tab = TAB_CONTACTS } DialerTab("Contacts", tab == TAB_CONTACTS, Modifier.width(140.dp)) { tab = TAB_CONTACTS }
@@ -0,0 +1,60 @@
package app.seb3thehacker.gearslip.car
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/**
* The text field the car keyboard is typing into, shared with the phone so a passenger can type
* on the phone's own keyboard instead. Whichever [CarKeyboard] is on screen registers here; the
* phone app shows the same text while one is open, and its edits go back through the car field's
* own callbacks, so the car screen can't tell which keyboard typed.
*/
object PhoneTyping {
class Field internal constructor(
val id: Int,
val text: String,
internal val onTextChange: (String) -> Unit,
internal val onSubmit: () -> Unit,
internal val onDismiss: (() -> Unit)?,
)
private val fieldFlow = MutableStateFlow<Field?>(null)
/** The car's open field, or null when no car keyboard is showing. */
val field: StateFlow<Field?> = fieldFlow.asStateFlow()
private var nextId = 0
internal fun open(text: String, onTextChange: (String) -> Unit, onSubmit: () -> Unit, onDismiss: (() -> Unit)?): Int {
val id = ++nextId
fieldFlow.value = Field(id, text, onTextChange, onSubmit, onDismiss)
return id
}
/** The car field's text changed, from either keyboard. */
internal fun carText(id: Int, text: String) {
val f = fieldFlow.value ?: return
if (f.id == id && f.text != text) fieldFlow.value = Field(id, text, f.onTextChange, f.onSubmit, f.onDismiss)
}
internal fun close(id: Int) {
if (fieldFlow.value?.id == id) fieldFlow.value = null
}
/** From the phone: typed text goes to the car field as if typed there. */
fun type(text: String) {
fieldFlow.value?.onTextChange?.invoke(text)
}
fun submit() {
fieldFlow.value?.onSubmit?.invoke()
}
/** Closes the car keyboard where it can be closed; elsewhere only the phone's field goes. */
fun dismiss() {
val f = fieldFlow.value ?: return
f.onDismiss?.invoke() ?: close(f.id)
}
}
@@ -138,29 +138,47 @@ internal fun PlayModeRow(
val repeatOn = repeatKnown && modes.repeat != PlaybackStateCompat.REPEAT_MODE_NONE val repeatOn = repeatKnown && modes.repeat != PlaybackStateCompat.REPEAT_MODE_NONE
val icon = size * 0.52f val icon = size * 0.52f
Row(modifier, horizontalArrangement = Arrangement.SpaceEvenly, verticalAlignment = Alignment.CenterVertically) { Row(modifier, horizontalArrangement = Arrangement.SpaceEvenly, verticalAlignment = Alignment.CenterVertically) {
GsIconButton( // An app's own like, shuffle or repeat button shows whether it's on only through its icon,
if (liked == true) Icons.Filled.Favorite else Icons.Filled.FavoriteBorder, // so that icon is drawn, as Android Auto does, and lit when its name says it's on.
if (liked == true) "Unlike" else "Like", // Gearslip's own stand in for apps that report a heart rating or shuffle and repeat modes.
media::toggleLike, val likeAction = now.likeAction
enabled = liked != null, if (likeAction != null) {
tone = if (liked == true) GsTone.Primary else GsTone.Neutral, CustomActionButton(likeAction, media, size, toggle = true)
size = size, iconSize = icon, } else {
) GsIconButton(
GsIconButton( if (liked == true) Icons.Filled.Favorite else Icons.Filled.FavoriteBorder,
MediaIcons.Shuffle, if (shuffleOn) "Shuffle off" else "Shuffle on", if (liked == true) "Unlike" else "Like",
media::toggleShuffle, media::toggleLike,
enabled = shuffleKnown || now.shuffleAction != null, enabled = liked != null,
tone = if (shuffleOn) GsTone.Primary else GsTone.Neutral, tone = if (liked == true) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon, size = size, iconSize = icon,
) )
GsIconButton( }
if (modes.repeat == PlaybackStateCompat.REPEAT_MODE_ONE) MediaIcons.RepeatOne else MediaIcons.Repeat, val shuffleAction = now.shuffleAction
"Repeat", if (shuffleAction != null) {
media::cycleRepeat, CustomActionButton(shuffleAction, media, size, toggle = true)
enabled = repeatKnown || now.repeatAction != null, } else {
tone = if (repeatOn) GsTone.Primary else GsTone.Neutral, GsIconButton(
size = size, iconSize = icon, MediaIcons.Shuffle, if (shuffleOn) "Shuffle off" else "Shuffle on",
) media::toggleShuffle,
enabled = shuffleKnown,
tone = if (shuffleOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
}
val repeatAction = now.repeatAction
if (repeatAction != null) {
CustomActionButton(repeatAction, media, size, toggle = true)
} else {
GsIconButton(
if (modes.repeat == PlaybackStateCompat.REPEAT_MODE_ONE) MediaIcons.RepeatOne else MediaIcons.Repeat,
"Repeat",
media::cycleRepeat,
enabled = repeatKnown,
tone = if (repeatOn) GsTone.Primary else GsTone.Neutral,
size = size, iconSize = icon,
)
}
now.otherActions.take(maxOther).forEach { CustomActionButton(it, media, size) } now.otherActions.take(maxOther).forEach { CustomActionButton(it, media, size) }
trailing() trailing()
} }
@@ -171,12 +189,12 @@ internal fun PlayModeRow(
* not Gearslip's, so it's loaded from there. Its name stands in if the icon won't load. * not Gearslip's, so it's loaded from there. Its name stands in if the icon won't load.
*/ */
@Composable @Composable
private fun CustomActionButton(action: CustomAction, media: CarMedia, size: Dp) { private fun CustomActionButton(action: CustomAction, media: CarMedia, size: Dp, toggle: Boolean = false) {
val icon = rememberActionIcon(action) val icon = rememberActionIcon(action)
GsIconBox( GsIconBox(
onClick = { media.custom(action) }, onClick = { media.custom(action) },
modifier = Modifier.size(size), modifier = Modifier.size(size),
colors = gsColors(GsTone.Neutral), colors = gsColors(if (toggle && action.on) GsTone.Primary else GsTone.Neutral),
) { ) {
if (icon != null) { if (icon != null) {
Icon(icon, action.name, Modifier.size(size * 0.52f)) Icon(icon, action.name, Modifier.size(size * 0.52f))
@@ -1,13 +1,15 @@
package app.seb3thehacker.gearslip.car package app.seb3thehacker.gearslip.car
import android.content.Context import android.content.Context
import app.seb3thehacker.gearslip.CertProvider
import app.seb3thehacker.gearslip.GearslipLog import app.seb3thehacker.gearslip.GearslipLog
import app.seb3thehacker.gearslip.mirror.PhoneApps import app.seb3thehacker.gearslip.mirror.PhoneApps
import java.util.concurrent.Executors import java.util.concurrent.Executors
/** /**
* Loads what the car screens show before anyone opens them: the car and media apps with their * Loads what the car screens show before anyone opens them: the car and media apps with their
* icons, the phone's launcher apps, the weather and the calendar. Each screen paints from these * icons, the phone's launcher apps, the weather and the calendar. Also unlocks the certificate,
* so a head unit's first message is answered at once. Each screen paints from these
* results at once and refreshes behind them, so nothing waits on the package manager, the * results at once and refreshes behind them, so nothing waits on the package manager, the
* network or the calendar provider. * network or the calendar provider.
* *
@@ -28,6 +30,7 @@ object Prefetch {
val app = context.applicationContext val app = context.applicationContext
worker.execute { worker.execute {
val start = System.currentTimeMillis() val start = System.currentTimeMillis()
step("certificate") { CertProvider.warm(app) }
step("launcher apps") { warmLauncherApps(app) } step("launcher apps") { warmLauncherApps(app) }
step("phone apps") { PhoneApps.warm(app) } step("phone apps") { PhoneApps.warm(app) }
step("weather and calendar") { warmDashboard(app) } step("weather and calendar") { warmDashboard(app) }
@@ -0,0 +1,43 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.RowScope
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.automirrored.filled.ArrowBack
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
import app.seb3thehacker.gearslip.car.theme.GsIconButton
/**
* The top of every screen reached from Home: a back button in the current theme, the title, and
* room on the right for the screen's own controls. A car touchscreen has no hardware back button,
* so this is the way out of a screen.
*/
@Composable
fun ScreenHeader(title: String, modifier: Modifier = Modifier, trailing: @Composable RowScope.() -> Unit = {}) {
val navigator = LocalCarNavigator.current
Row(
modifier.fillMaxWidth(),
verticalAlignment = Alignment.CenterVertically,
horizontalArrangement = Arrangement.spacedBy(12.dp),
) {
GsIconButton(Icons.AutoMirrored.Filled.ArrowBack, "Back", navigator::back)
Text(
title,
style = MaterialTheme.typography.headlineSmall,
fontWeight = FontWeight.Bold,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.weight(1f),
)
trailing()
}
}
@@ -2,6 +2,8 @@ package app.seb3thehacker.gearslip.car
import app.seb3thehacker.gearslip.car.theme.* import app.seb3thehacker.gearslip.car.theme.*
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.material.icons.filled.Close
import androidx.compose.material.icons.filled.Search
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.PaddingValues import androidx.compose.foundation.layout.PaddingValues
@@ -43,6 +45,7 @@ import androidx.car.app.model.MessageTemplate
import androidx.car.app.model.PaneTemplate import androidx.car.app.model.PaneTemplate
import androidx.car.app.model.Row as CarRow import androidx.car.app.model.Row as CarRow
import androidx.car.app.model.SearchTemplate import androidx.car.app.model.SearchTemplate
import androidx.compose.foundation.clickable
import androidx.car.app.model.Section import androidx.car.app.model.Section
import androidx.car.app.model.SectionedItemTemplate import androidx.car.app.model.SectionedItemTemplate
import androidx.car.app.model.TabTemplate import androidx.car.app.model.TabTemplate
@@ -236,6 +239,9 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
// initialSearchText would fight whatever is being typed. // initialSearchText would fight whatever is being typed.
var query by remember { mutableStateOf(template.initialSearchText.orEmpty()) } var query by remember { mutableStateOf(template.initialSearchText.orEmpty()) }
val delegate = template.searchCallbackDelegate val delegate = template.searchCallbackDelegate
// An app that opens on results ("navigate to Starbucks") asks for the keyboard to stay down,
// so the results aren't hidden under it. Tapping the field brings it up.
var typing by remember { mutableStateOf(template.isShowKeyboardByDefault) }
ContentSurface(modifier) { ContentSurface(modifier) {
// Back button and field share a row: on a 480px screen the keyboard and the results // Back button and field share a row: on a 480px screen the keyboard and the results
@@ -246,7 +252,15 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
horizontalArrangement = Arrangement.spacedBy(8.dp), horizontalArrangement = Arrangement.spacedBy(8.dp),
) { ) {
template.headerAction?.let { ActionButton(it) } template.headerAction?.let { ActionButton(it) }
SearchField(query, template.searchHint.orEmpty(), Modifier.weight(1f)) SearchField(
query,
template.searchHint.orEmpty(),
Modifier.weight(1f).clickable { typing = true },
onClear = {
query = ""
delegate.textChanged("")
},
)
ActionRow(template.actionStrip?.actions.orEmpty()) ActionRow(template.actionStrip?.actions.orEmpty())
} }
@@ -255,33 +269,73 @@ private fun SearchContent(template: SearchTemplate, modifier: Modifier) {
else ItemListColumn(template.itemList, modifier = Modifier.fillMaxSize()) else ItemListColumn(template.itemList, modifier = Modifier.fillMaxSize())
} }
CarKeyboard( if (typing) {
text = query, CarKeyboard(
onTextChange = { text = query,
query = it onTextChange = {
delegate.textChanged(it) query = it
}, delegate.textChanged(it)
onSubmit = { delegate.submitted(query) }, },
) onSubmit = { delegate.submitted(query) },
onDismiss = { typing = false },
)
}
} }
} }
/**
* The field the on-screen keyboard types into. It is the same height and shape as the round
* buttons beside it, so the header reads as one row of controls rather than a thin strip squeezed
* between two big buttons, and it carries a magnifier so it reads as search at a glance.
* [onClear] adds a clear button once something is typed; [searchIcon] is off for sign-in fields.
*/
@Composable @Composable
private fun SearchField(query: String, hint: String, modifier: Modifier = Modifier) { internal fun SearchField(
query: String,
hint: String,
modifier: Modifier = Modifier,
searchIcon: Boolean = true,
onClear: (() -> Unit)? = null,
) {
val muted = MaterialTheme.colorScheme.onSurfaceVariant
Surface( Surface(
color = MaterialTheme.colorScheme.surfaceContainerHighest, color = MaterialTheme.colorScheme.surfaceContainerHighest,
shape = RoundedCornerShape(10.dp), shape = androidx.compose.foundation.shape.CircleShape,
modifier = modifier.fillMaxWidth(), modifier = modifier.fillMaxWidth().height(ICON_BUTTON_SIZE),
) { ) {
Text( Row(
query.ifEmpty { hint.ifEmpty { "Search" } }, Modifier.padding(start = if (searchIcon) 16.dp else 22.dp, end = 6.dp),
style = ChromeType.body, verticalAlignment = Alignment.CenterVertically,
color = if (query.isEmpty()) MaterialTheme.colorScheme.onSurfaceVariant ) {
else MaterialTheme.colorScheme.onSurface, if (searchIcon) {
maxLines = 1, androidx.compose.material3.Icon(
overflow = TextOverflow.Ellipsis, androidx.compose.material.icons.Icons.Filled.Search,
modifier = Modifier.padding(horizontal = 14.dp, vertical = 8.dp), contentDescription = null,
) tint = muted,
modifier = Modifier.size(26.dp),
)
Spacer(Modifier.width(12.dp))
}
Text(
query.ifEmpty { hint.ifEmpty { "Search" } },
style = MaterialTheme.typography.titleMedium,
color = if (query.isEmpty()) muted else MaterialTheme.colorScheme.onSurface,
maxLines = 1,
overflow = TextOverflow.Ellipsis,
modifier = Modifier.weight(1f),
)
if (onClear != null && query.isNotEmpty()) {
GsIconButton(
androidx.compose.material.icons.Icons.Filled.Close,
"Clear",
onClear,
colors = GsColors(androidx.compose.ui.graphics.Color.Transparent, muted),
size = 42.dp,
iconSize = 22.dp,
shape = androidx.compose.foundation.shape.CircleShape,
)
}
}
} }
} }
@@ -344,7 +398,7 @@ private fun InputSignIn(method: InputSignInMethod, modifier: Modifier) {
val masked = method.inputType == InputSignInMethod.INPUT_TYPE_PASSWORD val masked = method.inputType == InputSignInMethod.INPUT_TYPE_PASSWORD
Column(modifier) { Column(modifier) {
SearchField(if (masked) "•".repeat(value.length) else value, method.hint.text()) SearchField(if (masked) "•".repeat(value.length) else value, method.hint.text(), searchIcon = false)
val error = method.errorMessage.text() val error = method.errorMessage.text()
if (error.isNotEmpty()) { if (error.isNotEmpty()) {
Text( Text(
@@ -3,8 +3,10 @@ package app.seb3thehacker.gearslip.car
import app.seb3thehacker.gearslip.car.theme.* import app.seb3thehacker.gearslip.car.theme.*
import androidx.compose.runtime.produceState import androidx.compose.runtime.produceState
import androidx.compose.runtime.remember import androidx.compose.runtime.remember
import androidx.compose.ui.focus.onFocusChanged
import androidx.compose.material3.LocalContentColor import androidx.compose.material3.LocalContentColor
import androidx.compose.ui.graphics.ColorFilter import androidx.compose.ui.graphics.ColorFilter
import androidx.compose.ui.graphics.luminance
import androidx.compose.ui.graphics.ImageBitmap import androidx.compose.ui.graphics.ImageBitmap
import androidx.compose.foundation.Image import androidx.compose.foundation.Image
import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.Dispatchers
@@ -29,6 +31,7 @@ import androidx.compose.foundation.lazy.grid.items
import androidx.compose.foundation.lazy.items import androidx.compose.foundation.lazy.items
import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.foundation.shape.RoundedCornerShape
import androidx.compose.material.icons.Icons import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Call
import androidx.compose.material.icons.filled.ArrowBack import androidx.compose.material.icons.filled.ArrowBack
import androidx.compose.material.icons.filled.Menu import androidx.compose.material.icons.filled.Menu
import androidx.compose.material.icons.filled.Warning import androidx.compose.material.icons.filled.Warning
@@ -231,7 +234,9 @@ internal fun RightEdgeControls(
// the first touch on one wakes them rather than pressing it. // the first touch on one wakes them rather than pressing it.
val awake = LocalMapControlsWake.current.awake val awake = LocalMapControlsWake.current.awake
val pan by CarServices.nav.panMode.collectAsState() val pan by CarServices.nav.panMode.collectAsState()
val visible = awake || pan // Kept up while the knob is on one of them, or the ring would fade out along with the button.
var holdsFocus by remember { mutableStateOf(false) }
val visible = awake || pan || holdsFocus
val alpha by androidx.compose.animation.core.animateFloatAsState( val alpha by androidx.compose.animation.core.animateFloatAsState(
targetValue = if (visible) 1f else 0f, targetValue = if (visible) 1f else 0f,
animationSpec = androidx.compose.animation.core.tween(if (visible) 150 else 500), animationSpec = androidx.compose.animation.core.tween(if (visible) 150 else 500),
@@ -240,6 +245,7 @@ internal fun RightEdgeControls(
Column( Column(
modifier modifier
.alpha(alpha) .alpha(alpha)
.onFocusChanged { holdsFocus = it.hasFocus }
.let { base -> .let { base ->
if (visible) base else base.pointerInput(Unit) { if (visible) base else base.pointerInput(Unit) {
awaitPointerEventScope { awaitPointerEventScope {
@@ -468,7 +474,10 @@ internal fun CarGlyph(icon: CarIcon?, modifier: Modifier = Modifier, standard: I
val white = remember(bitmap) { bitmap.isPlainWhite() } val white = remember(bitmap) { bitmap.isPlainWhite() }
// Untinted white artwork is a template-app habit from dark surfaces; it is coloured like // Untinted white artwork is a template-app habit from dark surfaces; it is coloured like
// text so it stays readable on a light one. // text so it stays readable on a light one.
val tint = icon?.tint?.color(dark, content) ?: content.takeIf { white } // An app's own tint is kept only if it can be seen: MAPS.ME asks for dark grey on dark
// buttons. Android Auto drops colours that fail contrast the same way.
val tint = icon?.tint?.color(dark, content)?.takeIf { it.readableOn(content) }
?: content.takeIf { white || icon?.tint != null }
Image( Image(
bitmap, contentDescription = null, modifier = modifier, bitmap, contentDescription = null, modifier = modifier,
colorFilter = tint?.let { ColorFilter.tint(it) }, colorFilter = tint?.let { ColorFilter.tint(it) },
@@ -587,16 +596,141 @@ internal fun PaneRows(pane: Pane?, modifier: Modifier = Modifier, autoStart: Boo
// Some apps pad a pane with a row of blank strings; drawn, it is just a gap. // Some apps pad a pane with a row of blank strings; drawn, it is just a gap.
val rows = pane?.rows.orEmpty().filterNot { it.isBlank() } val rows = pane?.rows.orEmpty().filterNot { it.isBlank() }
val actions = pane?.actions.orEmpty().filter { it.isDrawable() } val actions = pane?.actions.orEmpty().filter { it.isDrawable() }
val card = PlaceCard.of(rows)
Column(modifier) { Column(modifier) {
rows.forEach { RowItem(it, large = true) } card.heading?.let { PlaceHeading(it) }
if (card.buttons.isNotEmpty()) RowButtons(card.buttons, Modifier.padding(horizontal = 14.dp, vertical = 6.dp))
if (autoStart && actions.size == 1) { if (autoStart && actions.size == 1) {
AutoStartButton(actions.single(), Modifier.padding(horizontal = 14.dp, vertical = 10.dp)) AutoStartButton(actions.single(), Modifier.padding(horizontal = 14.dp, vertical = 10.dp))
} else { } else {
ActionRow(actions, Modifier.padding(horizontal = 14.dp, vertical = 10.dp), large = true) ActionRow(actions, Modifier.padding(horizontal = 14.dp, vertical = 10.dp), large = true)
} }
if (card.heading == null && card.buttons.isEmpty()) {
card.details.forEach { RowItem(it, large = true) }
} else {
card.details.forEach { DetailRow(it, card.heading?.title?.text().orEmpty()) }
}
} }
} }
/**
* A pane split the way a place page reads: a name on top, the things to do with it as buttons,
* then the facts. Apps send all three as plain rows, so the split is worked out from the rows'
* shape alone. A pane that doesn't fit the shape - all of it tappable, like a menu - gets no
* heading and no buttons, and draws exactly as a plain list of rows.
*/
private class PlaceCard(val heading: CarRow?, val buttons: List<CarRow>, val details: List<CarRow>) {
companion object {
fun of(rows: List<CarRow>): PlaceCard {
// A short tappable row with an icon and nothing under it is a button in all but name:
// MAPS.ME's "Let's go" and "Save".
val buttons = rows.filter {
it.onClickDelegate != null && it.toggle == null && it.image != null &&
it.texts.orEmpty().isEmpty() && it.title.text().length <= 24
}.takeIf { it.size in 1..4 && it.size < rows.size }.orEmpty()
// A first row that can't be tapped and has no icon is the place's name and kind.
val heading = rows.firstOrNull()?.takeIf {
buttons.isNotEmpty() && it.onClickDelegate == null && it.image == null && it.toggle == null
}
return PlaceCard(heading, buttons, rows - buttons.toSet() - listOfNotNull(heading).toSet())
}
}
}
@Composable
private fun PlaceHeading(row: CarRow) {
Column(Modifier.fillMaxWidth().padding(horizontal = 14.dp, vertical = 6.dp)) {
Text(row.title.text(), style = ChromeType.title, maxLines = 2, overflow = TextOverflow.Ellipsis)
row.texts.orEmpty().take(2).forEach {
Text(it.text(), style = ChromeType.body, color = MaterialTheme.colorScheme.onSurfaceVariant, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
}
/** Button-shaped rows side by side, the first one filled as the thing most drivers came to do. */
@Composable
private fun RowButtons(rows: List<CarRow>, modifier: Modifier) {
val scheme = MaterialTheme.colorScheme
Row(modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(8.dp)) {
rows.forEachIndexed { i, row ->
GsIconBox(
onClick = { row.onClickDelegate.click("row") },
modifier = Modifier.weight(1f),
colors = if (i == 0) GsColors(scheme.primary, scheme.onPrimary) else GsColors(scheme.surfaceContainerHighest, scheme.onSurface),
enabled = row.isEnabled,
shape = RoundedCornerShape(24.dp),
) {
Row(
Modifier.fillMaxWidth().padding(horizontal = 16.dp, vertical = 14.dp),
horizontalArrangement = Arrangement.Center,
verticalAlignment = Alignment.CenterVertically,
) {
CarGlyph(row.image, Modifier.size(24.dp))
Spacer(Modifier.width(8.dp))
Text(row.title.text(), style = ChromeType.body, maxLines = 1, overflow = TextOverflow.Ellipsis)
}
}
}
}
}
/**
* One fact about a place, tighter than a list row: a small label over its value, or just the
* value when the row has no label. A phone number gets a Call button that asks first.
*/
@Composable
private fun DetailRow(row: CarRow, placeName: String) {
val context = LocalContext.current
val texts = row.texts.orEmpty().map { it.text() }.filter { it.isNotBlank() }
val label = row.title.text().takeIf { texts.isNotEmpty() }
val values = texts.ifEmpty { listOf(row.title.text()) }
val number = values.firstOrNull { it.isPhoneNumber() }
val canCall = number != null &&
context.checkSelfPermission(android.Manifest.permission.CALL_PHONE) == android.content.pm.PackageManager.PERMISSION_GRANTED
Row(
Modifier
.fillMaxWidth()
.let { if (row.onClickDelegate != null && row.isEnabled) it.clickable { row.onClickDelegate.click("row") } else it }
.padding(horizontal = 14.dp, vertical = 6.dp),
verticalAlignment = Alignment.CenterVertically,
) {
row.image?.let {
CarGlyph(it, Modifier.size(24.dp))
Spacer(Modifier.width(12.dp))
}
Column(Modifier.weight(1f)) {
label?.let {
Text(it, style = MaterialTheme.typography.labelLarge, color = MaterialTheme.colorScheme.onSurfaceVariant, maxLines = 1)
}
values.take(3).forEach {
Text(it, style = ChromeType.body, maxLines = 2, overflow = TextOverflow.Ellipsis)
}
}
if (canCall) {
Spacer(Modifier.width(8.dp))
GsIconBox(
onClick = { CallPrompt.ask(placeName.ifBlank { number!! }, number!!) },
colors = GsColors(MaterialTheme.colorScheme.primary, MaterialTheme.colorScheme.onPrimary),
shape = RoundedCornerShape(24.dp),
) {
Row(Modifier.padding(horizontal = 16.dp, vertical = 10.dp), verticalAlignment = Alignment.CenterVertically) {
androidx.compose.material3.Icon(Icons.Filled.Call, null, Modifier.size(20.dp))
Spacer(Modifier.width(6.dp))
Text("Call", style = ChromeType.body)
}
}
}
ActionRow(row.actions.orEmpty())
}
}
/** A line that is only a phone number: digits with the usual separators, 7 to 15 digits long. */
private fun String.isPhoneNumber(): Boolean {
val t = trim()
if (!Regex("""^\+?[0-9 ().\-]+$""").matches(t)) return false
return t.count { it.isDigit() } in 7..15
}
@Composable @Composable
internal fun RowItem(row: CarRow, selection: RowSelection? = null, large: Boolean = false) { internal fun RowItem(row: CarRow, selection: RowSelection? = null, large: Boolean = false) {
val toggle = row.toggle val toggle = row.toggle
@@ -661,6 +795,17 @@ internal fun RowItem(row: CarRow, selection: RowSelection? = null, large: Boolea
/** A row's place in a single-choice list: whether it is the chosen one, and what picking it does. */ /** A row's place in a single-choice list: whether it is the chosen one, and what picking it does. */
internal class RowSelection(val selected: Boolean, val onPick: () -> Unit) internal class RowSelection(val selected: Boolean, val onPick: () -> Unit)
/**
* Whether this colour stands out against the background that [content] was chosen for: at least
* 3:1, the WCAG minimum for icons. The background is taken as the opposite of the content colour.
*/
private fun Color.readableOn(content: Color): Boolean {
val background = if (content.luminanceIsDark()) Color.White else Color(0xFF121212)
val a = luminance() + 0.05f
val b = background.luminance() + 0.05f
return maxOf(a, b) / minOf(a, b) >= 3f
}
internal fun Color.luminanceIsDark(): Boolean = internal fun Color.luminanceIsDark(): Boolean =
(red * 0.299f + green * 0.587f + blue * 0.114f) < 0.5f (red * 0.299f + green * 0.587f + blue * 0.114f) < 0.5f
@@ -680,5 +825,5 @@ private fun androidx.compose.ui.graphics.ImageBitmap.isPlainWhite(): Boolean {
return seen > 0 return seen > 0
} }
private val ICON_BUTTON_SIZE = 54.dp internal val ICON_BUTTON_SIZE = 54.dp
private val ICON_BUTTON_GLYPH = 30.dp private val ICON_BUTTON_GLYPH = 30.dp
@@ -28,7 +28,7 @@ fun VehicleDataScreen() {
val readings by CarSensors.readings.collectAsState() val readings by CarSensors.readings.collectAsState()
Column(Modifier.fillMaxSize().padding(horizontal = 24.dp, vertical = 16.dp)) { Column(Modifier.fillMaxSize().padding(horizontal = 24.dp, vertical = 16.dp)) {
Text("Vehicle data", style = MaterialTheme.typography.headlineMedium, fontWeight = FontWeight.Bold) ScreenHeader("Vehicle data")
Text( Text(
"Raw sensor channel from the connected head unit. Field meaning is a best guess - " + "Raw sensor channel from the connected head unit. Field meaning is a best guess - " +
"watch a row change while you do the thing (brake, drive, headlights) to confirm it.", "watch a row change while you do the thing (brake, drive, headlights) to confirm it.",
@@ -0,0 +1,94 @@
package app.seb3thehacker.gearslip.car
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material.icons.Icons
import androidx.compose.material.icons.filled.Close
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Surface
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.collectAsState
import androidx.compose.runtime.getValue
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp
import app.seb3thehacker.gearslip.car.theme.GsIconButton
import kotlinx.coroutines.delay
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
/** What the assistant understands, shown on the car screen when the driver says "help". */
object VoiceHelp {
private val _shown = MutableStateFlow(false)
val shown: StateFlow<Boolean> = _shown.asStateFlow()
fun show() { _shown.value = true }
fun hide() { _shown.value = false }
/** Say this, and this happens. Kept in step with [AssistantCommands]. */
val COMMANDS = listOf(
"Navigate to…" to "Directions there. Add \"with Organic Maps\" to pick the app",
"Play…" to "A song, artist, or album in the open music app",
"Pause · Resume · Next · Previous" to "Control the music",
"What's playing" to "The song and artist",
"Call…" to "A contact by name",
"Open…" to "Any app, like Browser, Phone, or Screen sharing",
"Exit" to "Back to the car's own screen",
"Go home · Open apps · Settings" to "Move around Gearslip",
"What time is it · Weather" to "The time, or the forecast",
"Help" to "This list",
)
}
/** The command list over the car screen: read it at a glance, tap the cross or wait to dismiss. */
@Composable
fun VoiceHelpOverlay(modifier: Modifier = Modifier) {
val shown by VoiceHelp.shown.collectAsState()
if (!shown) return
// A list left up would cover the map for the rest of the drive.
LaunchedEffect(Unit) {
delay(30_000)
VoiceHelp.hide()
}
Surface(
color = MaterialTheme.colorScheme.surfaceContainerHigh,
contentColor = MaterialTheme.colorScheme.onSurface,
shape = MaterialTheme.shapes.extraLarge,
tonalElevation = 6.dp,
shadowElevation = 8.dp,
modifier = modifier.padding(16.dp).widthIn(max = 640.dp),
) {
Column(Modifier.padding(20.dp)) {
Row(Modifier.fillMaxWidth()) {
Text(
"You can say",
style = MaterialTheme.typography.titleLarge,
fontWeight = FontWeight.Bold,
modifier = Modifier.weight(1f).padding(top = 6.dp),
)
GsIconButton(Icons.Filled.Close, "Close", VoiceHelp::hide)
}
// Full width, so a drag anywhere on the card scrolls, not just one on the words.
Column(
Modifier.fillMaxWidth().verticalScroll(rememberScrollState()).padding(top = 8.dp),
verticalArrangement = Arrangement.spacedBy(10.dp),
) {
VoiceHelp.COMMANDS.forEach { (say, does) ->
Column(Modifier.fillMaxWidth()) {
Text(say, style = MaterialTheme.typography.titleMedium, fontWeight = FontWeight.SemiBold)
Text(does, style = MaterialTheme.typography.bodyMedium, color = MaterialTheme.colorScheme.onSurfaceVariant)
}
}
}
}
}
}
@@ -41,7 +41,7 @@ object VoiceReply {
LISTENING, LISTENING,
/** /**
* Reading the reply back, then counting down to [State.until] while listening for * Reading the reply back, then counting down to [State.until] while listening for
* "cancel" or "change it". [State.until] is 0 while the countdown is held. * "cancel" or "change it". The countdown runs out whatever it hears.
*/ */
CONFIRMING, CONFIRMING,
SENT, SENT,
@@ -92,24 +92,37 @@ object VoiceReply {
} }
} }
/** "Do you want to reply?" A yes goes on to the reply; anything else ends quietly. */ /**
* "Do you want to reply?" A yes goes on to the reply; anything else ends quietly. The bar runs
* while it listens, so the card never looks stuck, and holds once the driver starts talking.
*/
private fun offer(id: Int) { private fun offer(id: Int) {
_state.value = _state.value.copy(phase = Phase.OFFERING, heard = "") _state.value = _state.value.copy(phase = Phase.OFFERING, heard = "", until = 0L, span = 0L, held = 0f)
CarAssistant.say("Do you want to reply?") { CarAssistant.say("Do you want to reply?") {
if (id != run) return@say if (id != run) return@say
val until = SystemClock.uptimeMillis() + OFFER_MS
_state.value = _state.value.copy(until = until, span = OFFER_MS)
// The recognizer gives up on its own; this only covers one that doesn't.
main.postAtTime({ if (id == run && _state.value.until != 0L) { CarAssistant.stopListening(); close() } }, until + OFFER_GRACE_MS)
CarAssistant.dictate( CarAssistant.dictate(
onPartial = { text -> if (id == run) _state.value = _state.value.copy(heard = text) }, onPartial = { text -> if (id == run) _state.value = _state.value.copy(heard = text) },
onSpeaking = {
if (id != run) return@dictate
val s = _state.value
val left = ((s.until - SystemClock.uptimeMillis()).toFloat() / s.span).coerceIn(0.05f, 1f)
_state.value = s.copy(until = 0L, span = 0L, held = left)
},
onResult = { text -> onResult = { text ->
if (id != run) return@dictate if (id != run) return@dictate
if (isYes(text)) { if (isYes(text)) {
_state.value = _state.value.copy(phase = Phase.ASKING, heard = "") _state.value = _state.value.copy(phase = Phase.ASKING, heard = "", until = 0L, span = 0L, held = 0f)
CarAssistant.say("Go ahead.") { if (id == run) listen(id) } CarAssistant.say("Go ahead.") { if (id == run) listen(id) }
} else { } else {
finish() close()
} }
}, },
// Heard nothing: leave the Reply button up for a moment, the same as after reading. // No answer counts as no: the message has been heard, so the card goes.
onError = { if (id == run) idleAfterRead(id) }, onError = { if (id == run) close() },
) )
} }
} }
@@ -180,43 +193,27 @@ object VoiceReply {
} }
} }
/** /** Sends in [ms]. */
* Sends in [ms]. After a hold the bar carries on from where it stopped instead of jumping back
* to full: the span is stretched so the bar still runs out exactly at [ms].
*/
private fun startCountdown(id: Int, ms: Long) { private fun startCountdown(id: Int, ms: Long) {
val sendAt = SystemClock.uptimeMillis() + ms val sendAt = SystemClock.uptimeMillis() + ms
val held = _state.value.held _state.value = _state.value.copy(until = sendAt, span = ms, held = 0f)
val span = if (held > 0f) (ms / held).toLong() else ms
_state.value = _state.value.copy(until = sendAt, span = span, held = 0f)
main.removeCallbacksAndMessages(SEND_TOKEN) main.removeCallbacksAndMessages(SEND_TOKEN)
main.postAtTime({ if (id == run) send() }, SEND_TOKEN, sendAt) main.postAtTime({ if (id == run) send() }, SEND_TOKEN, sendAt)
} }
/** /**
* Listens through the countdown for "cancel", "change it" or "send". The countdown holds from * Listens through the countdown for "cancel", "change it" or "send". The countdown never
* the first word, so a reply can't go out while the driver is still saying "cancel". * pauses: in a noisy car it held for road noise and the radio, and the reply never went.
*/ */
private fun listenForCommand(id: Int) { private fun listenForCommand(id: Int) {
CarAssistant.dictate( CarAssistant.dictate(
onPartial = {}, onPartial = {},
cue = false, cue = false,
onSpeaking = {
if (id != run) return@dictate
main.removeCallbacksAndMessages(SEND_TOKEN)
val s = _state.value
val left = if (s.until > 0L && s.span > 0L) {
((s.until - SystemClock.uptimeMillis()).toFloat() / s.span).coerceIn(0.05f, 1f)
} else {
s.held
}
_state.value = s.copy(until = 0L, span = 0L, held = left)
},
onResult = { text -> onResult = { text ->
if (id != run) return@dictate if (id != run) return@dictate
when (command(text)) { when (command(text)) {
Command.CANCEL -> { Command.CANCEL -> {
finish() close()
CarAssistant.say("Not sent.") {} CarAssistant.say("Not sent.") {}
} }
Command.CHANGE -> { Command.CHANGE -> {
@@ -225,11 +222,17 @@ object VoiceReply {
ask(id) ask(id)
} }
Command.SEND -> send() Command.SEND -> send()
// Talk that wasn't meant for us, a passenger or the radio: carry on counting. // Talk that wasn't meant for us, a passenger or the radio: keep listening, so a
null -> startCountdown(id, RESUME_MS) // "cancel" after it still lands before the countdown runs out.
null -> listenForCommand(id)
} }
}, },
onError = { if (id == run && _state.value.until == 0L) startCountdown(id, RESUME_MS) }, // Silence ends the recognizer before the countdown does: listen again until it sends.
onError = {
if (id != run) return@dictate
// A short gap, so a recognizer that fails at once can't spin.
main.postDelayed({ if (id == run) listenForCommand(id) }, RELISTEN_MS)
},
) )
} }
@@ -295,11 +298,20 @@ object VoiceReply {
} }
} }
/** Stops wherever it is. Nothing is sent. */ /** Stops wherever it is. Nothing is sent, and the message card closes. */
fun cancel() { fun cancel() {
if (!active) return if (!active) return
CarAssistant.stopListening() CarAssistant.stopListening()
CarAssistant.stopSpeaking() CarAssistant.stopSpeaking()
close()
}
/**
* Ends without sending and takes the message card with it. Without this the card's own popup
* came back with a fresh bar, as if the message had just arrived.
*/
private fun close() {
_state.value.target?.let { CarNotifications.dismissPopup(it.id) }
finish() finish()
} }
@@ -362,6 +374,8 @@ object VoiceReply {
private val SEND_TOKEN = Any() private val SEND_TOKEN = Any()
private const val ECHO_MS = 300L private const val ECHO_MS = 300L
/** After talk that wasn't a command, the reply waits this long again before it goes. */ /** How long "Do you want to reply?" waits for an answer: the start beep, then the engine's silence limit. */
private const val RESUME_MS = 3_000L private const val OFFER_MS = 6_500L
private const val OFFER_GRACE_MS = 1_500L
private const val RELISTEN_MS = 400L
} }
@@ -84,11 +84,13 @@ object Weather {
return return
} }
val data = runCatching { fetch(context, location) } val data = runCatching { fetch(context, location) }
.onFailure { GearslipLog.w("weather fetch failed: ${it.message}") } // Only the kind of error: some messages carry the request URL, and that holds the location.
.onFailure { GearslipLog.w("weather fetch failed: ${it.javaClass.simpleName}") }
.getOrNull() .getOrNull()
if (data != null) { if (data != null) {
flow.value = WeatherState.Ready(data) flow.value = WeatherState.Ready(data)
GearslipLog.i("weather: ${data.place} ${data.temp}${data.tempUnit} ${describe(data.code)}") // No place name: logs get shared in bug reports.
GearslipLog.i("weather: loaded")
} }
else if (flow.value !is WeatherState.Ready) flow.value = WeatherState.Unavailable("Could not reach the weather service.") else if (flow.value !is WeatherState.Ready) flow.value = WeatherState.Unavailable("Could not reach the weather service.")
} }
@@ -41,6 +41,7 @@ fun WeatherScreen() {
Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(horizontal = 24.dp, vertical = 16.dp), Modifier.fillMaxSize().verticalScroll(rememberScrollState()).padding(horizontal = 24.dp, vertical = 16.dp),
verticalArrangement = Arrangement.spacedBy(20.dp), verticalArrangement = Arrangement.spacedBy(20.dp),
) { ) {
ScreenHeader("Weather")
when (val s = state) { when (val s = state) {
WeatherState.Loading -> Text("Getting the weather…", style = MaterialTheme.typography.titleMedium) WeatherState.Loading -> Text("Getting the weather…", style = MaterialTheme.typography.titleMedium)
is WeatherState.Unavailable -> Text( is WeatherState.Unavailable -> Text(
@@ -10,6 +10,9 @@ import android.webkit.WebView
import android.webkit.WebViewClient import android.webkit.WebViewClient
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.ui.unit.dp
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable import androidx.compose.runtime.Composable
import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.DisposableEffect
import androidx.compose.runtime.getValue import androidx.compose.runtime.getValue
@@ -17,6 +20,7 @@ import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.remember import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue import androidx.compose.runtime.setValue
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import androidx.compose.ui.graphics.toArgb
import androidx.compose.ui.viewinterop.AndroidView import androidx.compose.ui.viewinterop.AndroidView
import app.seb3thehacker.gearslip.GearslipLog import app.seb3thehacker.gearslip.GearslipLog
import org.json.JSONObject import org.json.JSONObject
@@ -36,6 +40,7 @@ import org.json.JSONObject
fun WebApp(content: String) { fun WebApp(content: String) {
val navigator = LocalCarNavigator.current val navigator = LocalCarNavigator.current
val darkOutside = LocalDarkOutside.current val darkOutside = LocalDarkOutside.current
val ring = String.format("#%06X", MaterialTheme.colorScheme.primary.toArgb() and 0xFFFFFF)
var web by remember { mutableStateOf<WebView?>(null) } var web by remember { mutableStateOf<WebView?>(null) }
var fieldText by remember { mutableStateOf<String?>(null) } var fieldText by remember { mutableStateOf<String?>(null) }
@@ -53,6 +58,8 @@ fun WebApp(content: String) {
} }
Column(Modifier.fillMaxSize()) { Column(Modifier.fillMaxSize()) {
// Back steps through the page's history first (the interceptor above), then leaves.
ScreenHeader("Browser", Modifier.padding(horizontal = 16.dp, vertical = 8.dp))
AndroidView( AndroidView(
modifier = Modifier.weight(1f).fillMaxSize(), modifier = Modifier.weight(1f).fillMaxSize(),
factory = { context -> factory = { context ->
@@ -61,7 +68,7 @@ fun WebApp(content: String) {
settings.domStorageEnabled = true settings.domStorageEnabled = true
settings.loadWithOverviewMode = true settings.loadWithOverviewMode = true
settings.useWideViewPort = true settings.useWideViewPort = true
webViewClient = LoggingClient() webViewClient = LoggingClient(ring)
addJavascriptInterface( addJavascriptInterface(
FieldFocusBridge(onFocus = { fieldText = it }, onBlur = { fieldText = null }), FieldFocusBridge(onFocus = { fieldText = it }, onBlur = { fieldText = null }),
"GearslipField", "GearslipField",
@@ -95,10 +102,11 @@ fun WebApp(content: String) {
} }
} }
private class LoggingClient : WebViewClient() { private class LoggingClient(private val ring: String) : WebViewClient() {
override fun onPageFinished(view: WebView?, url: String?) { override fun onPageFinished(view: WebView?, url: String?) {
GearslipLog.i("car web page loaded: $url") GearslipLog.i("car web page loaded: $url")
view?.evaluateJavascript(FOCUS_TRACKING_JS, null) view?.evaluateJavascript(FOCUS_TRACKING_JS, null)
view?.evaluateJavascript(focusRingJs(ring), null)
} }
// onPageFinished fires for error pages too, so without this a blocked load (missing // onPageFinished fires for error pages too, so without this a blocked load (missing
@@ -166,6 +174,20 @@ private const val SUBMIT_FIELD_JS = """
})(); })();
""" """
/**
* The same ring the rest of the car UI shows, on whatever link or field the knob is on. Many
* sites hide the browser's own focus outline, which leaves a knob driver with no idea where they are.
*/
private fun focusRingJs(color: String) = """
(function() {
if (document.getElementById('gearslip-focus-ring')) return;
var style = document.createElement('style');
style.id = 'gearslip-focus-ring';
style.textContent = ':focus-visible { outline: 3px solid $color !important; outline-offset: 2px !important; }';
(document.head || document.documentElement).appendChild(style);
})();
"""
/** Reports focus/blur on text inputs, textareas and contenteditable elements to [FieldFocusBridge]. */ /** Reports focus/blur on text inputs, textareas and contenteditable elements to [FieldFocusBridge]. */
private const val FOCUS_TRACKING_JS = """ private const val FOCUS_TRACKING_JS = """
(function() { (function() {
@@ -315,6 +315,7 @@ fun GsIconBox(
shape: Shape = gsShape(), shape: Shape = gsShape(),
/** Held down, as a keycap theme draws it - for a toggle or the current screen's nav button. */ /** Held down, as a keycap theme draws it - for a toggle or the current screen's nav button. */
latched: Boolean = false, latched: Boolean = false,
/** Also marks the control with the theme's hold marker, so the hold can be found. */
onLongClick: (() -> Unit)? = null, onLongClick: (() -> Unit)? = null,
content: @Composable BoxScope.() -> Unit, content: @Composable BoxScope.() -> Unit,
) { ) {
@@ -327,6 +328,7 @@ fun GsIconBox(
.gsSurface(c.container, shape, c.border, pressed, latched), .gsSurface(c.container, shape, c.border, pressed, latched),
contentAlignment = Alignment.Center, contentAlignment = Alignment.Center,
) { ) {
if (onLongClick != null && enabled) GsHoldMarker(gsLook().holdMarker, c.container, shape, source)
CompositionLocalProvider(LocalContentColor provides c.content) { content() } CompositionLocalProvider(LocalContentColor provides c.content) { content() }
} }
} }
@@ -0,0 +1,183 @@
package app.seb3thehacker.gearslip.car.theme
import androidx.compose.animation.core.Animatable
import androidx.compose.animation.core.LinearEasing
import androidx.compose.animation.core.tween
import androidx.compose.foundation.Canvas
import androidx.compose.foundation.interaction.InteractionSource
import androidx.compose.foundation.interaction.collectIsPressedAsState
import androidx.compose.foundation.layout.BoxScope
import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.compose.runtime.remember
import androidx.compose.ui.Modifier
import androidx.compose.ui.geometry.Offset
import androidx.compose.ui.graphics.Brush
import androidx.compose.ui.graphics.Color
import androidx.compose.ui.graphics.Path
import androidx.compose.ui.graphics.Shape
import androidx.compose.foundation.shape.CornerBasedShape
import androidx.compose.ui.graphics.drawscope.DrawScope
import androidx.compose.ui.graphics.drawscope.scale
import androidx.compose.ui.graphics.drawscope.translate
import androidx.compose.ui.graphics.lerp
import androidx.compose.ui.platform.LocalViewConfiguration
import androidx.compose.ui.unit.dp
import kotlin.math.PI
import kotlin.math.cos
import kotlin.math.min
import kotlin.math.sin
/**
* How a control says it also does something when held. [GsIconBox] draws it on its own for any
* control given an onLongClick, so callers never place it by hand.
*/
enum class HoldMarkerStyle {
/** The top layer peels back from the top-right corner, further while held. Needs depth. */
PEEL,
/** A plain dot in the top-right corner. It doesn't react to a hold. */
DOT,
/** Nothing: the hold stays undiscovered. For themes that mark it some other way. */
NONE,
}
/**
* The hold marker for one control, drawn under its content. [container] is the control's own
* colour, so the peel's flap matches whatever it sits on. Round and pill-shaped controls get the
* dot whatever the theme says: a circle has no corner to peel.
*/
private const val SHOW_HOLD_DOT = false
@Composable
internal fun BoxScope.GsHoldMarker(style: HoldMarkerStyle, container: Color, shape: Shape, press: InteractionSource) {
if (style == HoldMarkerStyle.NONE) return
val pressed by press.collectIsPressedAsState()
val timeout = LocalViewConfiguration.current.longPressTimeoutMillis.toInt()
val lift = remember { Animatable(0f) }
if (style == HoldMarkerStyle.PEEL) {
LaunchedEffect(pressed) {
if (pressed) lift.animateTo(1f, tween(timeout, easing = LinearEasing)) else lift.animateTo(0f, tween(220))
}
}
val ink = MaterialTheme.colorScheme.onSurfaceVariant
val accent = MaterialTheme.colorScheme.primaryContainer
val onAccent = MaterialTheme.colorScheme.onPrimaryContainer
Canvas(Modifier.matchParentSize()) {
val side = min(size.width, size.height)
val r = (shape as? CornerBasedShape)?.topEnd?.toPx(size, this) ?: 0f
val round = r >= side * 0.4f
// The dot is off until it's redesigned: flat themes and round controls show nothing.
if (style == HoldMarkerStyle.DOT || round) {
if (SHOW_HOLD_DOT) drawHoldDot(r, side, ink)
} else {
// Drawn for the bottom-right corner, then flipped up to the top-right.
scale(1f, -1f, pivot = center) {
drawPeel(r, side, container, accent, onAccent, lift.value)
}
}
}
}
/** Depth of a corner's curve along its diagonal: a fold shallower than this only shaves the curve. */
private fun curveDepth(r: Float) = r * (2f - 1.41421f)
private fun DrawScope.drawHoldDot(r: Float, side: Float, ink: Color) {
val dot = (side * 0.06f).coerceIn(5.dp.toPx(), 7.dp.toPx())
// Tucked into the corner, clear of its curve.
val inset = r * (1f - 0.7071f) + (side * 0.05f).coerceIn(4.dp.toPx(), 6.dp.toPx()) + dot / 2
drawCircle(ink.copy(alpha = 0.6f), radius = dot / 2, center = Offset(size.width - inset, inset))
}
/** The part of a rounded rectangle beyond the fold line x + y = c, as a closed outline. */
private fun cornerRegion(w: Float, h: Float, r: Float, c: Float): List<Offset> {
// Walk the bottom-right boundary: down the right edge, round the corner, along the bottom.
val reach = (h + w - c) + 1f
val edge = ArrayList<Offset>()
edge += Offset(w, h - maxOf(r, reach))
if (r > 0f) {
for (i in 0..24) {
val a = (i / 24f) * (PI / 2).toFloat()
edge += Offset(w - r + r * cos(a), h - r + r * sin(a))
}
} else {
edge += Offset(w, h)
}
edge += Offset(w - maxOf(r, reach), h)
// Keep what lies beyond the fold, adding the exact crossing points.
val out = ArrayList<Offset>()
for (i in 0 until edge.size - 1) {
val p = edge[i]
val q = edge[i + 1]
val ps = p.x + p.y - c
val qs = q.x + q.y - c
if (ps >= 0) out += p
if ((ps >= 0) != (qs >= 0)) {
val t = ps / (ps - qs)
out += Offset(p.x + (q.x - p.x) * t, p.y + (q.y - p.y) * t)
}
}
val last = edge.last()
if (last.x + last.y - c >= 0) out += last
return out
}
private fun pathOf(points: List<Offset>): Path = Path().apply {
if (points.isEmpty()) return@apply
moveTo(points[0].x, points[0].y)
for (p in points.drop(1)) lineTo(p.x, p.y)
close()
}
/**
* The top layer peeling back from the corner, showing a second layer underneath with the same
* rounded corner, so the control's outline never changes. Sized from the control: a launcher
* tile folds about 27dp at rest and 64dp held, a nav bar button far less.
*/
private fun DrawScope.drawPeel(r: Float, side: Float, face: Color, accent: Color, onAccent: Color, lift: Float) {
val w = size.width
val h = size.height
val rest = curveDepth(r) + (side * 0.1f).coerceIn(4.dp.toPx(), 11.dp.toPx())
val held = rest + min(37.dp.toPx(), side * 0.35f)
val e = rest + (held - rest) * lift
val c = w + h - e
val region = cornerRegion(w, h, r, c)
if (region.size < 3) return
val mid = Offset(w - e / 2, h - e / 2)
// 1. The second layer, turning the accent colour as the hold completes.
drawPath(pathOf(region), lerp(lerp(face, Color.Black, 0.26f), accent, lift))
// The lifted edge of the top layer shades the second layer just beside it.
drawPath(
pathOf(region),
Brush.linearGradient(
listOf(Color.Black.copy(alpha = 0.35f), Color.Transparent),
start = mid,
end = mid + Offset(e * 0.25f, e * 0.25f),
),
)
// 2. "More" waits on the second layer, readable once the peel is well back.
if (lift > 0.35f && side > 80.dp.toPx()) {
val a = ((lift - 0.35f) / 0.65f).coerceIn(0f, 1f)
val centre = Offset(w - e * 0.3f, h - e * 0.3f)
val gap = 5.5.dp.toPx()
for (k in -1..1) {
drawCircle(onAccent.copy(alpha = a), radius = 2.2.dp.toPx(), center = centre + Offset(k * gap * 0.71f, -k * gap * 0.71f))
}
}
// 3. The flap: the same corner folded back over the face, its rounded tip and all.
val flap = region.map { Offset(c - it.y, c - it.x) }
translate(-1.5.dp.toPx(), -1.5.dp.toPx()) {
drawPath(pathOf(flap), Color.Black.copy(alpha = 0.30f))
}
drawPath(
pathOf(flap),
Brush.linearGradient(
listOf(lerp(face, Color.White, 0.30f), lerp(face, Color.White, 0.08f)),
start = Offset(c - h, c - w),
end = mid,
),
)
}
@@ -59,6 +59,11 @@ data class ControlLook(
val pressedDim: Float = 0.22f, val pressedDim: Float = 0.22f,
/** Scale while pressed, 1 for none. */ /** Scale while pressed, 1 for none. */
val pressedScale: Float = 0.95f, val pressedScale: Float = 0.95f,
/**
* How a control with a press-and-hold action says so. The peel needs a raised face to lift
* from; a flat look takes the dot.
*/
val holdMarker: HoldMarkerStyle = HoldMarkerStyle.PEEL,
/** Replaces this look when the UI is light, for effects that only work on one background. */ /** Replaces this look when the UI is light, for effects that only work on one background. */
val light: ControlLook? = null, val light: ControlLook? = null,
) )
@@ -67,7 +72,7 @@ object GsThemes {
val Keycap = GsTheme( val Keycap = GsTheme(
id = "keycap", id = "keycap",
name = "Keycap", name = "Keycap",
control = ControlLook(lip = 4.dp, pressedScale = 1f), control = ControlLook(lip = 4.dp, pressedScale = 1f, holdMarker = HoldMarkerStyle.PEEL),
) )
val Tactile = GsTheme( val Tactile = GsTheme(
@@ -83,6 +88,7 @@ object GsThemes {
insetAlpha = 0.30f, insetAlpha = 0.30f,
pressedDim = 0.12f, pressedDim = 0.12f,
pressedScale = 1f, pressedScale = 1f,
holdMarker = HoldMarkerStyle.PEEL,
// A wide soft shadow bands once the car's video stream compresses it on a light // A wide soft shadow bands once the car's video stream compresses it on a light
// background, so light mode gets a tight one and leans on the bevel instead. // background, so light mode gets a tight one and leans on the bevel instead.
light = ControlLook( light = ControlLook(
@@ -95,6 +101,7 @@ object GsThemes {
insetAlpha = 0.30f, insetAlpha = 0.30f,
pressedDim = 0.12f, pressedDim = 0.12f,
pressedScale = 1f, pressedScale = 1f,
holdMarker = HoldMarkerStyle.PEEL,
), ),
), ),
) )
@@ -102,10 +109,10 @@ object GsThemes {
val Flat = GsTheme( val Flat = GsTheme(
id = "flat", id = "flat",
name = "Flat", name = "Flat",
control = ControlLook(), control = ControlLook(holdMarker = HoldMarkerStyle.DOT),
) )
val default = Tactile val default = Flat
val builtIn = listOf(Tactile, Keycap, Flat) val builtIn = listOf(Tactile, Keycap, Flat)
fun byId(id: String?): GsTheme = builtIn.firstOrNull { it.id == id } ?: default fun byId(id: String?): GsTheme = builtIn.firstOrNull { it.id == id } ?: default
@@ -0,0 +1,98 @@
package app.seb3thehacker.gearslip.host
import android.os.SystemClock
import android.text.Spanned
import androidx.car.app.model.CarText
import androidx.car.app.model.DistanceSpan
import androidx.car.app.model.ItemList
import androidx.car.app.model.ListTemplate
import androidx.car.app.model.PaneTemplate
import androidx.car.app.model.PlaceListMapTemplate
import androidx.car.app.model.Row
import androidx.car.app.model.SearchTemplate
import androidx.car.app.model.Template
import androidx.car.app.navigation.model.MapTemplate
import androidx.car.app.navigation.model.MapWithContentTemplate
import androidx.car.app.navigation.model.NavigationTemplate
import androidx.car.app.navigation.model.PlaceListNavigationTemplate
import androidx.car.app.navigation.model.RoutePreviewNavigationTemplate
import app.seb3thehacker.gearslip.GearslipLog
/**
* "Navigate to X" the way Android Auto does it: straight onto a route, no list to pick from.
*
* Android Auto looks the place up on Google's servers first. Gearslip stays on the phone, so it
* asks the map app to search, then taps the nearest result in the app's list for the driver.
* It stops at the app's start screen, so the driver sees where it's going and taps Start. It
* gives up after [WINDOW_MS], or as soon as the driver taps anything themselves.
*/
internal class AutoRoute {
private val log = GearslipLog.tagged("route")
private var until = 0L
private var pickedRow = false
/** A destination was just sent to the map app. */
fun arm() {
armedAt = SystemClock.elapsedRealtime()
until = armedAt + WINDOW_MS
pickedRow = false
}
fun disarm() {
until = 0L
}
private var armedAt = 0L
private val armed get() = SystemClock.elapsedRealtime() < until && driverTappedAt < armedAt
/** Each screen the map app sends while armed: tap the next step if it's there. */
@Suppress("DEPRECATION") // Apps still send the older navigation templates.
fun onTemplate(template: Template) {
if (!armed) return
when (template) {
is NavigationTemplate -> disarm()
// The start screen: the driver takes it from here.
is RoutePreviewNavigationTemplate -> arrived()
is MapWithContentTemplate -> onTemplate(template.contentTemplate)
is PaneTemplate -> arrived()
is MapTemplate -> if (template.pane != null) arrived() else template.itemList?.let(::onList)
is SearchTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
is ListTemplate -> if (!template.isLoading) template.singleList?.let(::onList)
is PlaceListMapTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
is PlaceListNavigationTemplate -> if (!template.isLoading) template.itemList?.let(::onList)
}
}
/** Search results: the nearest place, or the first when the app gives no distances. */
private fun onList(list: ItemList) {
if (pickedRow) return
val places = list.items.filterIsInstance<Row>()
.filter { it.isEnabled && it.onClickDelegate != null && !it.isBrowsable }
if (places.isEmpty()) return
val nearest = places.minByOrNull { it.meters() ?: Double.MAX_VALUE } ?: return
pickedRow = true
log.i("picking the nearest of ${places.size} results")
nearest.onClickDelegate.click(AUTO_LABEL)
}
private fun arrived() {
if (pickedRow) log.i("at the start screen")
disarm()
}
/** The first distance the app attached to this row's title or text lines. */
private fun Row.meters(): Double? = (listOfNotNull(title) + texts).firstNotNullOfOrNull { it.distanceMeters() }
private fun CarText.distanceMeters(): Double? {
val chars = toCharSequence() as? Spanned ?: return null
return chars.getSpans(0, chars.length, DistanceSpan::class.java).firstOrNull()?.distance?.meters()
}
companion object {
/** Taps sent by this class carry this label, so they don't count as the driver's. */
const val AUTO_LABEL = "auto-route"
/** When the driver last tapped one of the app's buttons; that hands control back. */
@Volatile var driverTappedAt = 0L
private const val WINDOW_MS = 30_000L
}
}
@@ -78,5 +78,7 @@ object CarAppCatalog {
CarAppService.CATEGORY_SETTINGS_APP, CarAppService.CATEGORY_SETTINGS_APP,
CarAppService.CATEGORY_IOT_APP, CarAppService.CATEGORY_IOT_APP,
CarAppService.CATEGORY_WEATHER_APP, CarAppService.CATEGORY_WEATHER_APP,
// Newer apps declare MEDIA, but our AndroidX 1.7 dependency has no constant for it.
"androidx.car.app.category.MEDIA",
) )
} }
@@ -1,6 +1,7 @@
package app.seb3thehacker.gearslip.host package app.seb3thehacker.gearslip.host
import app.seb3thehacker.gearslip.car.CarEnvironment import app.seb3thehacker.gearslip.car.CarEnvironment
import app.seb3thehacker.gearslip.car.CarToasts
import android.content.ComponentName import android.content.ComponentName
import android.content.Context import android.content.Context
import android.content.Intent import android.content.Intent
@@ -9,6 +10,7 @@ import android.content.res.Configuration
import android.graphics.Rect import android.graphics.Rect
import android.location.Location import android.location.Location
import android.net.Uri import android.net.Uri
import android.os.Build
import android.os.Handler import android.os.Handler
import android.os.IBinder import android.os.IBinder
import android.os.Looper import android.os.Looper
@@ -80,6 +82,8 @@ class CarAppConnection(private val context: Context) {
val template: StateFlow<Template?> = _template.asStateFlow() val template: StateFlow<Template?> = _template.asStateFlow()
private val main = Handler(Looper.getMainLooper()) private val main = Handler(Looper.getMainLooper())
/** Captured by each host Binder so callbacks queued by a replaced app cannot show a toast. */
private var toastOwner = Any()
private var carApp: ICarApp? = null private var carApp: ICarApp? = null
private var appManager: IAppManager? = null private var appManager: IAppManager? = null
@@ -139,7 +143,11 @@ class CarAppConnection(private val context: Context) {
// Replaced by another connect (or a disconnect) while waiting on the location service. // Replaced by another connect (or a disconnect) while waiting on the location service.
if (binding !== connection) return@bind if (binding !== connection) return@bind
var bindError: Throwable? = null var bindError: Throwable? = null
val flags = Context.BIND_AUTO_CREATE or LocationKeepAlive.BIND_INCLUDE_CAPABILITIES // BIND_ALLOW_ACTIVITY_STARTS lets the app start its own location service while
// Gearslip's car window is showing. Since Android 14 a bound app only inherits that
// from a visible host that opts in, and MapQuest crashes as navigation starts without it.
val flags = Context.BIND_AUTO_CREATE or LocationKeepAlive.BIND_INCLUDE_CAPABILITIES or
(if (Build.VERSION.SDK_INT >= 34) Context.BIND_ALLOW_ACTIVITY_STARTS else 0)
val bound = runCatching { val bound = runCatching {
// Without BIND_FOREGROUND_SERVICE, an app bound by a foreground service (which is // Without BIND_FOREGROUND_SERVICE, an app bound by a foreground service (which is
// all Gearslip is once the phone screen leaves it) only reaches "important // all Gearslip is once the phone screen leaves it) only reaches "important
@@ -215,6 +223,7 @@ class CarAppConnection(private val context: Context) {
fun disconnect() = disconnect(keepLocation = false) fun disconnect() = disconnect(keepLocation = false)
private fun disconnect(keepLocation: Boolean) { private fun disconnect(keepLocation: Boolean) {
clearToasts()
lent?.let { sendSurfaceDestroyed(it) } lent?.let { sendSurfaceDestroyed(it) }
lent = null lent = null
val carApp = this.carApp val carApp = this.carApp
@@ -231,6 +240,10 @@ class CarAppConnection(private val context: Context) {
this.carApp = null this.carApp = null
appManager = null appManager = null
navigationManager = null navigationManager = null
navigating = false
main.removeCallbacks(sendReroute)
reroute = null
autoRoute.disarm()
_panMode.value = false _panMode.value = false
surfaceCallback = null surfaceCallback = null
component = null component = null
@@ -281,34 +294,66 @@ class CarAppConnection(private val context: Context) {
private fun onHandshakeAccepted() { private fun onHandshakeAccepted() {
val app = carApp ?: return val app = carApp ?: return
GearslipLog.i("host: handshake accepted; creating the app") GearslipLog.i("host: handshake accepted; creating the app")
val intent = pendingDestination?.let { navigateIntent(it) } val intent = pendingDestination?.let { autoRoute.arm(); navigateIntent(it) }
?: Intent(Intent.ACTION_MAIN).setComponent(component) ?: Intent(Intent.ACTION_MAIN).setComponent(component)
pendingDestination = null pendingDestination = null
call("onAppCreate", onValue = { onCreated() }) { call("onAppCreate", onValue = { onCreated() }) {
app.onAppCreate(carHost, intent, carConfiguration(), it) app.onAppCreate(createCarHost(toastOwner), intent, carConfiguration(), it)
} }
} }
/** /**
* "navigate to X" deep link, the same one Android Auto's own Assistant integration sends - * "navigate to X" deep link, the same one Android Auto's own Assistant integration sends -
* [ACTION_NAVIGATE] with the destination as a `geo:` search query, rather than anything * [ACTION_NAVIGATE] with the destination as a `geo:0,0` search query, rather than anything
* app-specific. * app-specific. The app searches near its own location; [AutoRoute] then picks the nearest
* result. Coordinates in the link would mean "go exactly here": Organic Maps routed to the
* phone's own position when it was given them.
*/ */
private fun navigateIntent(destination: String) = private fun navigateIntent(destination: String): Intent =
Intent(CarContext.ACTION_NAVIGATE, Uri.parse("geo:0,0?q=" + Uri.encode(destination))) Intent(CarContext.ACTION_NAVIGATE, Uri.parse("geo:0,0?q=" + Uri.encode(destination)))
.setComponent(component) .setComponent(component)
/** /**
* Sends a fresh destination to an app that's already running, instead of tearing it down * Sends a fresh destination to an app that's already running, instead of tearing it down
* and rebuilding the whole session - see [app.seb3thehacker.gearslip.car.CarServices.connectNav]. * and rebuilding the whole session - see [app.seb3thehacker.gearslip.car.CarServices.connectNav].
*
* An app already on a route ends it first: Organic Maps ignored a new destination mid-route
* and kept the old directions. The new one goes once the app says the route ended, or after
* [REROUTE_WAIT_MS] if it never does.
*/ */
fun navigateTo(destination: String) { fun navigateTo(destination: String) {
if (carApp == null) return
main.removeCallbacks(sendReroute)
if (!navigating) {
reroute = null
sendDestination(destination)
return
}
reroute = destination
stopNavigating()
main.postDelayed(sendReroute, REROUTE_WAIT_MS)
}
private fun sendDestination(destination: String) {
val app = carApp ?: return val app = carApp ?: return
autoRoute.arm()
call("onNewIntent", onValue = {}) { call("onNewIntent", onValue = {}) {
app.onNewIntent(navigateIntent(destination), it) app.onNewIntent(navigateIntent(destination), it)
} }
} }
private val autoRoute = AutoRoute()
/** A destination waiting for the current route to end; see [navigateTo]. */
private var reroute: String? = null
private val sendReroute = Runnable {
reroute?.let { reroute = null; sendDestination(it) }
}
/** Whether the app has said it's on a route, between navigationStarted and navigationEnded. */
@Volatile var navigating = false
private set
private fun onCreated() { private fun onCreated() {
val app = carApp ?: return val app = carApp ?: return
GearslipLog.i("host: created; starting") GearslipLog.i("host: created; starting")
@@ -380,6 +425,7 @@ class CarAppConnection(private val context: Context) {
GearslipLog.i("host: template = ${template.javaClass.simpleName}") GearslipLog.i("host: template = ${template.javaClass.simpleName}")
_template.value = template _template.value = template
update { it.copy(template = template.javaClass.simpleName) } update { it.copy(template = template.javaClass.simpleName) }
autoRoute.onTemplate(template)
}) { manager.getTemplate(it) } }) { manager.getTemplate(it) }
} }
@@ -455,28 +501,28 @@ class CarAppConnection(private val context: Context) {
private val _surfaceEpoch = MutableStateFlow(0) private val _surfaceEpoch = MutableStateFlow(0)
/** Bumped when the lent surface turns out to be dead, so the stage builds a fresh one. */ /** Bumped to throw the lent surface away, so the stage builds a fresh one. */
val surfaceEpoch: StateFlow<Int> = _surfaceEpoch.asStateFlow() val surfaceEpoch: StateFlow<Int> = _surfaceEpoch.asStateFlow()
/** /**
* Lends the app its surface again: called when the map comes back into view. An app that * Called as the map comes back into view: replaces the surface only if it died while the map
* draws through its own VirtualDisplay (Organic Maps does) can be left pointing at a surface * was covered. A live one is left alone, because every hand-over costs the app its map - some
* that has since died, and draws black until it's told about a live one. Taking the surface * (MapQuest) rebuild theirs from a world view and fly back to the car's position.
* back and lending it again repoints it; a surface that has itself died is replaced. *
* Always a new surface, never the dead one lent again: an app that draws through its own
* VirtualDisplay (Flitsmeister does) builds its new display on a re-lent surface before
* releasing the old, and the old one's disconnect then takes the shared BufferQueue's
* connection with it - the map freezes on its last frame for good. The stage's key on
* [surfaceEpoch] swaps in a new SurfaceView, which lends itself as it comes up.
*/ */
fun refreshSurface() { fun replaceDeadSurface() {
main.post { main.post {
val current = lent ?: return@post val current = lent ?: return@post
if (!current.surface.isValid) { if (current.surface.isValid) return@post
GearslipLog.w("host: the lent surface died; building a new one") GearslipLog.w("host: the lent surface died; building a new one")
lent = null lent = null
sendSurfaceDestroyed(current)
_surfaceEpoch.value = _surfaceEpoch.value + 1
return@post
}
GearslipLog.i("host: lending the surface again as the map comes back")
sendSurfaceDestroyed(current) sendSurfaceDestroyed(current)
sendSurfaceAvailable() _surfaceEpoch.value = _surfaceEpoch.value + 1
} }
} }
@@ -504,9 +550,9 @@ class CarAppConnection(private val context: Context) {
} }
// The app draws for the whole surface; the visible area is what isn't covered by host // The app draws for the whole surface; the visible area is what isn't covered by host
// chrome, and the stable area is the part that never gets covered. We keep our chrome // chrome, and the stable area is the part that never gets covered. Both depend on the
// to a top bar, so both are the surface minus that bar - reported by the UI via // template (see visibleAreaFor) and are reported by the UI via [reportAreas] once it
// [reportAreas] once it has measured. Until then the whole surface is fair game. // has measured. Until then the whole surface is fair game.
val whole = Rect(0, 0, surface.width, surface.height) val whole = Rect(0, 0, surface.width, surface.height)
reportAreas(whole, whole) reportAreas(whole, whole)
@@ -558,14 +604,19 @@ class CarAppConnection(private val context: Context) {
// --- host interfaces the app calls back into ----------------------------------------- // --- host interfaces the app calls back into -----------------------------------------
private val appHost = object : IAppHost.Stub() { private fun createAppHost(owner: Any) = object : IAppHost.Stub() {
override fun invalidate() { override fun invalidate() {
GearslipLog.i("host: app invalidated its template") GearslipLog.i("host: app invalidated its template")
main.post { requestTemplate() } main.post { requestTemplate() }
} }
override fun showToast(text: CharSequence?, duration: Int) { override fun showToast(text: CharSequence?, duration: Int) {
GearslipLog.i("host: toast \"$text\"") val message = text?.toString() ?: return
main.post {
if (toastOwner !== owner) return@post
GearslipLog.i("host: toast \"$message\"")
CarToasts.show(owner, message, duration)
}
} }
/** /**
@@ -587,10 +638,24 @@ class CarAppConnection(private val context: Context) {
override fun openMicrophone(request: Bundleable?): Bundleable? = null override fun openMicrophone(request: Bundleable?): Bundleable? = null
} }
private val _trip = MutableStateFlow<androidx.car.app.navigation.model.Trip?>(null)
/** The route as the app tells clusters about it: steps and how far to each. Null when not navigating. */
val trip: StateFlow<androidx.car.app.navigation.model.Trip?> = _trip.asStateFlow()
private val navigationHost = object : INavigationHost.Stub() { private val navigationHost = object : INavigationHost.Stub() {
override fun navigationStarted() = GearslipLog.i("host: navigation started") override fun navigationStarted() {
override fun navigationEnded() = GearslipLog.i("host: navigation ended") GearslipLog.i("host: navigation started")
override fun updateTrip(trip: Bundleable?) = Unit navigating = true
}
override fun navigationEnded() {
GearslipLog.i("host: navigation ended")
navigating = false
_trip.value = null
main.post { if (reroute != null) { main.removeCallbacks(sendReroute); sendReroute.run() } }
}
override fun updateTrip(trip: Bundleable?) {
_trip.value = runCatching { trip?.get() as? androidx.car.app.navigation.model.Trip }.getOrNull()
}
} }
/** /**
@@ -656,7 +721,9 @@ class CarAppConnection(private val context: Context) {
override fun updateSuggestions(suggestions: Bundleable?) = Unit override fun updateSuggestions(suggestions: Bundleable?) = Unit
} }
private val carHost = object : ICarHost.Stub() { private fun createCarHost(owner: Any) = object : ICarHost.Stub() {
private val appHost = createAppHost(owner)
override fun getHost(type: String?): IBinder? = when (type) { override fun getHost(type: String?): IBinder? = when (type) {
CarContext.APP_SERVICE -> appHost.asBinder() CarContext.APP_SERVICE -> appHost.asBinder()
CarContext.NAVIGATION_SERVICE -> navigationHost.asBinder() CarContext.NAVIGATION_SERVICE -> navigationHost.asBinder()
@@ -745,10 +812,16 @@ class CarAppConnection(private val context: Context) {
} }
private fun fail(phase: Phase, detail: String) { private fun fail(phase: Phase, detail: String) {
clearToasts()
GearslipLog.e("host: $detail") GearslipLog.e("host: $detail")
update { it.copy(phase = phase, detail = detail) } update { it.copy(phase = phase, detail = detail) }
} }
private fun clearToasts() {
CarToasts.clear(toastOwner)
toastOwner = Any()
}
private fun update(block: (Status) -> Status) { private fun update(block: (Status) -> Status) {
_status.value = block(_status.value) _status.value = block(_status.value)
} }
@@ -757,6 +830,9 @@ class CarAppConnection(private val context: Context) {
/** The newest template API this host claims to understand. */ /** The newest template API this host claims to understand. */
const val HOST_API_LEVEL = CarAppApiLevels.LEVEL_8 const val HOST_API_LEVEL = CarAppApiLevels.LEVEL_8
/** How long a new destination waits for the old route to end; see [navigateTo]. */
const val REROUTE_WAIT_MS = 1_500L
/** Context.BIND_FOREGROUND_SERVICE; the constant itself is hidden from the SDK. */ /** Context.BIND_FOREGROUND_SERVICE; the constant itself is hidden from the SDK. */
const val BIND_FOREGROUND_SERVICE = 0x04000000 const val BIND_FOREGROUND_SERVICE = 0x04000000
} }
Loaded 100 of 149 files, more files were not shown because too many files have changed in this diff. Show more