Author SHA1 Message Date
Seb3thehackerandClaude Sonnet 5 48be745659 README: explain the screen-share indicator for audio
Gearslip captures playback, not the screen, but Android's consent
dialog and persistent indicator look identical either way. Note why
the indicator appears, and correct the adjacent no-adb claim to
mention the two optional audio commands as the sole exception.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 00:18:39 -07:00
Seb3thehackerandClaude Sonnet 5 7e4b4d6b99 Send media audio to the car without a per-drive prompt
The head unit offers a media sink, but Android answers Setup with an
empty reply instead of a Config message. AudioLink now falls back to
the advertised sink after 2.5s. AudioCaptureService captures by audio
usage, not by app, so switching media apps costs nothing further and
one grant covers a whole drive.

A one-time adb command (PROJECT_MEDIA) removes the consent dialog
itself; the setup screen now includes it as a copyable step, next to
the existing mic-permission step.

Capture counts as a screen share, so Android 15 hides notification
contents while it runs. ScreenShareGuard lifts that protection for
the length of the capture, with a second adb grant
(WRITE_SECURE_SETTINGS), and restores it afterward, including after a
crash.

The phone keeps playing what it captures, so AudioCaptureService also
mutes STREAM_MUSIC for the drive and restores it on every exit path.

Adds a debug bench (BenchAudioSink, CarPreviewActivity --ez audio
true) for testing the audio path without a car, and a unit test that
rebuilds the real head unit's ServiceDiscoveryResponse from a logged
session to confirm the media and guidance sinks are found correctly.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-22 00:11:59 -07:00
16 changed files with 621 additions and 53 deletions

No files matched your search

+8 -1
View File
@@ -23,6 +23,11 @@ than Google's own can talk to a car head unit.
tested against several real navigation and media apps. tested against several real navigation and media apps.
- **Media playback works.** Gearslip lets you browse and control car-enabled media apps - **Media playback works.** Gearslip lets you browse and control car-enabled media apps
from the car screen, with lyrics and playback state, over the same host connection. from the car screen, with lyrics and playback state, over the same host connection.
- **Audio reaches the car by capturing playback, not by mirroring the screen.** Android
requires a screen-share style consent dialog and indicator for this, even though
Gearslip never reads the screen; that grant is the only way a non-privileged app can
tap another app's audio. The consent normally repeats every drive; the app's own setup
steps include two optional adb commands that reduce it to a one-time grant.
- **A real head unit needs a certificate that Google controls.** A head unit accepts only - **A real head unit needs a certificate that Google controls.** A head unit accepts only
a phone identity that chains to Google's Automotive Link certificate authority, and a phone identity that chains to Google's Automotive Link certificate authority, and
Google alone issues those certificates. Gearslip does not ship one. Without it, Google alone issues those certificates. Gearslip does not ship one. Without it,
@@ -30,7 +35,9 @@ than Google's own can talk to a car head unit.
connection. A leaked certificate exists that older head units accept; newer firmware connection. A leaked certificate exists that older head units accept; newer firmware
and Google's reference tooling reject it. and Google's reference tooling reject it.
- **Gearslip needs no adb, root, or Shizuku.** Every feature relies on an ordinary - **Gearslip needs no adb, root, or Shizuku.** Every feature relies on an ordinary
Android permission or an on-device consent dialog. Android permission or an on-device consent dialog. The two adb commands mentioned
above, for audio, are the sole exception: both are optional, and both only remove a
dialog that the driver would otherwise see once per drive.
## Building ## Building
@@ -0,0 +1,62 @@
package app.seb3thehacker.gearslip
import app.seb3thehacker.gearslip.audio.CarAudio
import kotlin.math.abs
/**
* Debug-only stand-in for the head unit's media audio sink.
*
* The car end of the audio path can only be exercised in a car, but everything before it - the
* setting, the consent dialog, the capture service, and whether the media app allows itself to be
* heard at all - is phone-side and can be tested at a desk. This sink takes the place of
* [AudioLink] so that path runs to completion, and reports what arrived.
*
* adb shell am start -n app.seb3thehacker.gearslip/.CarPreviewActivity \
* --ei w 800 --ei h 480 --ez audio true --ei snaps 10
*
* A healthy run logs a rising byte count and a peak well above zero. A byte count that rises with
* a peak of zero means the app refused to be captured.
*/
class BenchAudioSink(override val sampleRate: Int = 48_000, override val channels: Int = 2) : CarAudio.Link {
private var chunks = 0
private var bytes = 0L
private var peak = 0
private var reportedAt = 0L
override fun start() {
chunks = 0
bytes = 0
peak = 0
reportedAt = System.currentTimeMillis()
GearslipLog.i("bench sink: started - pretending to be a ${sampleRate}Hz x$channels car sink")
}
override fun stop() {
GearslipLog.i("bench sink: stopped after $chunks chunks / $bytes bytes")
}
override fun write(pcm: ByteArray, size: Int) {
chunks++
bytes += size
for (i in 0 until size - 1 step 2) {
val sample = (pcm[i].toInt() and 0xFF) or (pcm[i + 1].toInt() shl 8)
peak = maxOf(peak, abs(sample.toShort().toInt()))
}
val now = System.currentTimeMillis()
if (now - reportedAt < REPORT_MS) return
reportedAt = now
val seconds = bytes / (sampleRate.toDouble() * channels * 2)
GearslipLog.i(
"bench sink: %d chunks / %d bytes (%.1fs of audio), peak %.3f%s".format(
chunks, bytes, seconds, peak / 32768f,
if (peak == 0) " - SILENT, the app is not letting itself be captured" else "",
),
)
peak = 0
}
companion object {
private const val REPORT_MS = 2_000L
}
}
@@ -9,6 +9,9 @@ import android.os.Handler
import android.os.Looper import android.os.Looper
import android.view.MotionEvent import android.view.MotionEvent
import androidx.activity.ComponentActivity import androidx.activity.ComponentActivity
import androidx.activity.result.contract.ActivityResultContracts
import app.seb3thehacker.gearslip.audio.AudioCaptureService
import app.seb3thehacker.gearslip.audio.CarAudio
import app.seb3thehacker.gearslip.car.CarEnvironment import app.seb3thehacker.gearslip.car.CarEnvironment
import app.seb3thehacker.gearslip.car.CarSettings import app.seb3thehacker.gearslip.car.CarSettings
import app.seb3thehacker.gearslip.car.CarUi import app.seb3thehacker.gearslip.car.CarUi
@@ -31,6 +34,28 @@ class CarPreviewActivity : ComponentActivity() {
private var reader: ImageReader? = null private var reader: ImageReader? = null
@Volatile private var latest: Bitmap? = null @Volatile private var latest: Bitmap? = null
private val audioPermission = registerForActivityResult(ActivityResultContracts.RequestPermission()) { granted ->
if (granted) askAudioCapture() else CarAudio.denied()
}
private val audioConsent = registerForActivityResult(
androidx.activity.result.contract.ActivityResultContracts.StartActivityForResult(),
) { result ->
val data = result.data
if (result.resultCode == RESULT_OK && data != null) {
AudioCaptureService.start(this, result.resultCode, data)
} else {
GearslipLog.w("preview: audio capture consent declined")
CarAudio.denied()
}
}
private fun askAudioCapture() {
val manager = getSystemService(android.media.projection.MediaProjectionManager::class.java)
runCatching { audioConsent.launch(manager.createScreenCaptureIntent()) }
.onFailure { GearslipLog.e("preview: could not ask for audio capture", it); CarAudio.denied() }
}
override fun onCreate(savedInstanceState: Bundle?) { override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState) super.onCreate(savedInstanceState)
GearslipLog.init(this) GearslipLog.init(this)
@@ -58,6 +83,22 @@ class CarPreviewActivity : ComponentActivity() {
val p = ScreenProjector(this).also { projector = it } val p = ScreenProjector(this).also { projector = it }
p.start(imageReader.surface, w, h, dpi) { CarUi() } p.start(imageReader.surface, w, h, dpi) { CarUi() }
// Stand in for the car's media sink, so the phone half of the audio path runs for real.
if (intent.getBooleanExtra("audio", false)) {
CarAudio.onConsentRequested = {
runOnUiThread {
if (checkSelfPermission(android.Manifest.permission.RECORD_AUDIO) ==
android.content.pm.PackageManager.PERMISSION_GRANTED
) {
askAudioCapture()
} else {
audioPermission.launch(android.Manifest.permission.RECORD_AUDIO)
}
}
}
CarAudio.attach(BenchAudioSink())
}
GearslipLog.i("preview: displayId=${p.displayId}") GearslipLog.i("preview: displayId=${p.displayId}")
if (intent.getBooleanExtra("hide", false)) p.setOverlayVisible(false) if (intent.getBooleanExtra("hide", false)) p.setOverlayVisible(false)
@@ -123,6 +164,8 @@ class CarPreviewActivity : ComponentActivity() {
super.onDestroy() super.onDestroy()
projector?.stop() projector?.stop()
reader?.close() reader?.close()
CarAudio.onConsentRequested = null
CarAudio.release()
CarEnvironment.stop() CarEnvironment.stop()
} }
} }
+17 -1
View File
@@ -1,5 +1,6 @@
<?xml version="1.0" encoding="utf-8"?> <?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"> <manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<!-- The projected WebView loads real pages; without this every load fails silently. --> <!-- The projected WebView loads real pages; without this every load fails silently. -->
<uses-permission android:name="android.permission.INTERNET" /> <uses-permission android:name="android.permission.INTERNET" />
@@ -23,6 +24,21 @@
<!-- Capturing a media app's playback (to send to the car) is gated on this by the platform. --> <!-- Capturing a media app's playback (to send to the car) is gated on this by the platform. -->
<uses-permission android:name="android.permission.RECORD_AUDIO" /> <uses-permission android:name="android.permission.RECORD_AUDIO" />
<!--
Playback capture copies a stream rather than moving it, so the phone keeps playing the
music it is sending to the car. Silencing the phone's own speaker for the drive needs this.
-->
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<!--
Audio capture counts as a screen share, and Android hides notifications during one. The
protection can be switched off for the drive, but only with this, granted once over adb
(pm grant). Without it Gearslip works the same and notifications stay hidden.
-->
<uses-permission
android:name="android.permission.WRITE_SECURE_SETTINGS"
tools:ignore="ProtectedPermissions" />
<!-- <!--
Declared, not just requested. Templated apps decide whether to talk to a host by asking Declared, not just requested. Templated apps decide whether to talk to a host by asking
whether it holds this permission (HostValidator.isValidHost checks it before falling back whether it holds this permission (HostValidator.isValidHost checks it before falling back
@@ -112,6 +112,14 @@ class GearslipActivity : ComponentActivity(), Projection {
if (checkSelfPermission(android.Manifest.permission.READ_CALENDAR) != if (checkSelfPermission(android.Manifest.permission.READ_CALENDAR) !=
android.content.pm.PackageManager.PERMISSION_GRANTED android.content.pm.PackageManager.PERMISSION_GRANTED
) calendarPermission.launch(android.Manifest.permission.READ_CALENDAR) ) calendarPermission.launch(android.Manifest.permission.READ_CALENDAR)
// Playback capture needs the microphone even though it never records one. Ask for it here,
// at the kerb, rather than the first time a song plays: a permission dialog that appears
// mid-drive is one the driver has to read and answer at the wheel, and missing it is
// indistinguishable from the car link being broken.
if (CarSettings.pipeAudio.value &&
checkSelfPermission(android.Manifest.permission.RECORD_AUDIO) !=
android.content.pm.PackageManager.PERMISSION_GRANTED
) audioPermission.launch(android.Manifest.permission.RECORD_AUDIO)
enableEdgeToEdge() enableEdgeToEdge()
window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON) window.addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON)
@@ -119,6 +127,11 @@ class GearslipActivity : ComponentActivity(), Projection {
GearslipLog.installCrashHandler() GearslipLog.installCrashHandler()
GearslipLog.i("Gearslip ready") GearslipLog.i("Gearslip ready")
CarSettings.init(this) CarSettings.init(this)
// A capture that died without tidying up would leave notifications exposed to every
// later screen share. Nothing is capturing if the service is not running, so undo it.
if (!app.seb3thehacker.gearslip.audio.AudioCaptureService.running) {
app.seb3thehacker.gearslip.audio.ScreenShareGuard.restore(this)
}
Prefetch.warm(this) Prefetch.warm(this)
CarEnvironment.setPhoneTheme(resources.configuration) CarEnvironment.setPhoneTheme(resources.configuration)
@@ -51,6 +51,7 @@ class GearslipRunner(
@Volatile private var dropped = 0 @Volatile private var dropped = 0
@Volatile private var inputChannelId: Int = -1 @Volatile private var inputChannelId: Int = -1
@Volatile private var audioLink: AudioLink? = null @Volatile private var audioLink: AudioLink? = null
private var audioMessagesSeen = 0
private var touchWidth = 0 private var touchWidth = 0
private var touchHeight = 0 private var touchHeight = 0
private var touchesSeen = 0 private var touchesSeen = 0
@@ -142,6 +143,16 @@ class GearslipRunner(
} else if (frame.channel == inputChannelId) { } else if (frame.channel == inputChannelId) {
onInputMessage(messageId, body) onInputMessage(messageId, body)
} else if (frame.channel == audioLink?.channelId) { } else if (frame.channel == audioLink?.channelId) {
// The audio channel is the one still being reverse-engineered, and its first few
// replies are what tell us why. Raw bytes for those, then quiet.
if (audioMessagesSeen < AUDIO_TRACE_MESSAGES) {
audioMessagesSeen++
log.i(
"audio frame #$audioMessagesSeen: channel=${frame.channel} " +
"encrypted=${frame.encrypted} raw=${frame.payload.size}B decrypted=${message.size}B",
)
log.hex(" audio raw", message, limit = 64)
}
audioLink?.onMessage(messageId, body) audioLink?.onMessage(messageId, body)
} else { } else {
log.i("ignoring message id $messageId on channel ${frame.channel}") log.i("ignoring message id $messageId on channel ${frame.channel}")
@@ -360,8 +371,16 @@ class GearslipRunner(
} }
val media = sinks.firstOrNull { it.streamType == STREAM_MEDIA } ?: run { val media = sinks.firstOrNull { it.streamType == STREAM_MEDIA } ?: run {
log.w("no media audio sink advertised - media apps will play on the phone only") log.w("no media audio sink advertised - media apps will play on the phone only")
SessionReport.audio(
"no MEDIA sink; head unit offered ${sinks.joinToString { it.streamName }.ifEmpty { "none" }}",
)
return return
} }
val first = media.configs.firstOrNull()
SessionReport.audio(
"MEDIA sink on channel ${media.serviceId}, ${media.codecName}, " +
"${first?.sampleRate ?: 48_000}Hz/${first?.bits ?: 16}bit/x${first?.channels ?: 2}",
)
audioLink = AudioLink( audioLink = AudioLink(
media, media,
sendOnChannel = { id, body -> send(id, body, encrypted = true, channel = media.serviceId) }, sendOnChannel = { id, body -> send(id, body, encrypted = true, channel = media.serviceId) },
@@ -799,6 +818,7 @@ class GearslipRunner(
const val ACTION_DOWN = 0 const val ACTION_DOWN = 0
const val ACTION_UP = 1 const val ACTION_UP = 1
const val ACK_STALL_MS = 2_000L const val ACK_STALL_MS = 2_000L
const val AUDIO_TRACE_MESSAGES = 6
const val PERIODIC_KEYFRAME_MS = 5_000L const val PERIODIC_KEYFRAME_MS = 5_000L
} }
} }
@@ -42,6 +42,7 @@ object SessionReport {
@Volatile private var headUnit = "" @Volatile private var headUnit = ""
@Volatile private var profile = "" @Volatile private var profile = ""
@Volatile private var video = "" @Volatile private var video = ""
@Volatile private var audio = ""
@Volatile private var lastStream = "" @Volatile private var lastStream = ""
@Volatile private var lastState = "" @Volatile private var lastState = ""
private var printed = false private var printed = false
@@ -68,6 +69,7 @@ object SessionReport {
fun headUnit(info: String) { headUnit = info } fun headUnit(info: String) { headUnit = info }
fun profile(name: String?) { profile = name ?: "none matched" } fun profile(name: String?) { profile = name ?: "none matched" }
fun video(summary: String) { video = summary } fun video(summary: String) { video = summary }
fun audio(summary: String) { audio = summary }
fun stream(summary: String) { lastStream = summary } fun stream(summary: String) { lastStream = summary }
/** Records the first failure only: later ones are usually fallout from it. */ /** Records the first failure only: later ones are usually fallout from it. */
@@ -107,6 +109,7 @@ object SessionReport {
append(row("auth status", authStatus?.let { "$it (${authName(it)})" } ?: "")) append(row("auth status", authStatus?.let { "$it (${authName(it)})" } ?: ""))
append("\n") append("\n")
append(row("video", video)) append(row("video", video))
append(row("audio", audio))
append(row("last stream stat", lastStream)) append(row("last stream stat", lastStream))
append("=".repeat(60)) append("=".repeat(60))
} }
@@ -24,16 +24,51 @@ import kotlin.math.abs
* Playback capture is the one public route to another app's sound, and it hangs off a * Playback capture is the one public route to another app's sound, and it hangs off a
* MediaProjection - so, as with [app.seb3thehacker.gearslip.mirror.MirrorService], the consent * MediaProjection - so, as with [app.seb3thehacker.gearslip.mirror.MirrorService], the consent
* result is carried into a foreground service of type mediaProjection before it can be redeemed. * result is carried into a foreground service of type mediaProjection before it can be redeemed.
* Capture is scoped to the media app's uid: notification pings and other apps stay on the phone. * Capture is scoped to media playback: notification pings, calls and alarms stay on the phone.
* The grant lasts as long as this service does, which is as long as the car is connected, so the
* driver answers the consent dialog once per drive rather than once per app.
*/ */
class AudioCaptureService : Service() { class AudioCaptureService : Service() {
@Volatile private var reading = false @Volatile private var reading = false
private var projection: MediaProjection? = null private var projection: MediaProjection? = null
private var thread: Thread? = null private var thread: Thread? = null
private var mutedPhone = false
/**
* Silence the phone's own speaker while the car is being fed.
*
* Playback capture copies the stream instead of moving it, so without this the music plays in
* two places at once. Muting the stream does not affect what is captured - the tap sits before
* the volume the speaker gets - so the car still receives full-level audio.
*/
private fun mutePhone() {
if (mutedPhone) return
runCatching {
getSystemService(android.media.AudioManager::class.java)
.adjustStreamVolume(android.media.AudioManager.STREAM_MUSIC, android.media.AudioManager.ADJUST_MUTE, 0)
mutedPhone = true
GearslipLog.i("audio: muted the phone's speaker for the drive")
}.onFailure { GearslipLog.w("audio: could not mute the phone: ${it.message}") }
}
private fun unmutePhone() {
if (!mutedPhone) return
mutedPhone = false
runCatching {
getSystemService(android.media.AudioManager::class.java)
.adjustStreamVolume(android.media.AudioManager.STREAM_MUSIC, android.media.AudioManager.ADJUST_UNMUTE, 0)
GearslipLog.i("audio: gave the phone's speaker back")
}.onFailure { GearslipLog.w("audio: could not unmute the phone: ${it.message}") }
}
override fun onBind(intent: Intent?): IBinder? = null override fun onBind(intent: Intent?): IBinder? = null
override fun onCreate() {
super.onCreate()
running = true
}
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int { override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
if (intent?.action == ACTION_STOP) { if (intent?.action == ACTION_STOP) {
stopSelf() stopSelf()
@@ -49,6 +84,7 @@ class AudioCaptureService : Service() {
} }
startForeground(NOTIFICATION_ID, notification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION) startForeground(NOTIFICATION_ID, notification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION)
ScreenShareGuard.lift(this)
projection = try { projection = try {
getSystemService(MediaProjectionManager::class.java).getMediaProjection(resultCode, data) getSystemService(MediaProjectionManager::class.java).getMediaProjection(resultCode, data)
} catch (t: Throwable) { } catch (t: Throwable) {
@@ -74,13 +110,14 @@ class AudioCaptureService : Service() {
private fun beginCapture(token: MediaProjection) { private fun beginCapture(token: MediaProjection) {
val rate = CarAudio.sampleRate val rate = CarAudio.sampleRate
val channels = CarAudio.channels val channels = CarAudio.channels
val uid = CarAudio.uid
val record = try { val record = try {
// Matched by usage, not by app. A uid filter would have to be rebuilt every time the
// driver changed media app, and rebuilding means a fresh consent dialog - so the
// filter is what the sound is for, and any app playing media reaches the car.
val config = AudioPlaybackCaptureConfiguration.Builder(token) val config = AudioPlaybackCaptureConfiguration.Builder(token)
.addMatchingUsage(AudioAttributes.USAGE_MEDIA) .addMatchingUsage(AudioAttributes.USAGE_MEDIA)
.addMatchingUsage(AudioAttributes.USAGE_GAME) .addMatchingUsage(AudioAttributes.USAGE_GAME)
.addMatchingUsage(AudioAttributes.USAGE_UNKNOWN) .addMatchingUsage(AudioAttributes.USAGE_UNKNOWN)
.apply { if (uid >= 0) addMatchingUid(uid) }
.build() .build()
val mask = if (channels == 1) AudioFormat.CHANNEL_IN_MONO else AudioFormat.CHANNEL_IN_STEREO val mask = if (channels == 1) AudioFormat.CHANNEL_IN_MONO else AudioFormat.CHANNEL_IN_STEREO
val format = AudioFormat.Builder() val format = AudioFormat.Builder()
@@ -106,17 +143,25 @@ class AudioCaptureService : Service() {
val buffer = ByteArray(CHUNK_BYTES) val buffer = ByteArray(CHUNK_BYTES)
var peak = 0 var peak = 0
var heardAt = System.currentTimeMillis() var heardAt = System.currentTimeMillis()
val startedAt = heardAt
var everHeard = false
var warnedSilent = false
try { try {
record.startRecording() record.startRecording()
GearslipLog.i("audio: capturing uid=$uid at ${rate}Hz x$channels") GearslipLog.i("audio: capturing media playback at ${rate}Hz x$channels")
mutePhone()
CarAudio.captureStarted() CarAudio.captureStarted()
while (reading) { while (reading) {
val n = record.read(buffer, 0, buffer.size) val n = record.read(buffer, 0, buffer.size)
if (n <= 0) break if (n <= 0) {
GearslipLog.e("audio: AudioRecord.read returned $n - capture has stopped")
break
}
for (i in 0 until n - 1 step 2) { for (i in 0 until n - 1 step 2) {
val sample = (buffer[i].toInt() and 0xFF) or (buffer[i + 1].toInt() shl 8) val sample = (buffer[i].toInt() and 0xFF) or (buffer[i + 1].toInt() shl 8)
peak = maxOf(peak, abs(sample.toShort().toInt())) peak = maxOf(peak, abs(sample.toShort().toInt()))
} }
if (peak > 0) everHeard = true
val now = System.currentTimeMillis() val now = System.currentTimeMillis()
val level = peak / 32768f val level = peak / 32768f
if (now - heardAt >= LEVEL_LOG_MS) { if (now - heardAt >= LEVEL_LOG_MS) {
@@ -124,6 +169,16 @@ class AudioCaptureService : Service() {
heardAt = now heardAt = now
peak = 0 peak = 0
} }
// Capture never fails for an app that opts out; it just hands back digital
// silence. Say so, because the symptom is identical to a broken car link.
if (!everHeard && !warnedSilent && now - startedAt >= SILENCE_MS) {
warnedSilent = true
GearslipLog.w(
"audio: ${SILENCE_MS / 1000}s of capture and every sample is zero. " +
"Either nothing is playing, or the app does not allow its audio " +
"to be captured.",
)
}
CarAudio.deliver(buffer, n, level) CarAudio.deliver(buffer, n, level)
} }
} catch (t: Throwable) { } catch (t: Throwable) {
@@ -131,6 +186,7 @@ class AudioCaptureService : Service() {
} finally { } finally {
runCatching { record.stop() } runCatching { record.stop() }
record.release() record.release()
unmutePhone()
CarAudio.captureStopped() CarAudio.captureStopped()
} }
}, "gearslip-audio").apply { isDaemon = true; start() } }, "gearslip-audio").apply { isDaemon = true; start() }
@@ -140,8 +196,13 @@ class AudioCaptureService : Service() {
super.onDestroy() super.onDestroy()
reading = false reading = false
runCatching { thread?.join(500) } runCatching { thread?.join(500) }
// Belt and braces: a phone left muted after the drive is a bug the driver blames on their
// phone, not on Gearslip, so unmute on every way out of here.
unmutePhone()
runCatching { projection?.stop() } runCatching { projection?.stop() }
projection = null projection = null
ScreenShareGuard.restore(this)
running = false
} }
private fun notification(): Notification { private fun notification(): Notification {
@@ -173,9 +234,14 @@ class AudioCaptureService : Service() {
/** About 21 ms of 48 kHz stereo: small enough for lip-sync, big enough not to flood the link. */ /** About 21 ms of 48 kHz stereo: small enough for lip-sync, big enough not to flood the link. */
private const val CHUNK_BYTES = 4096 private const val CHUNK_BYTES = 4096
private const val LEVEL_LOG_MS = 3_000L private const val LEVEL_LOG_MS = 3_000L
private const val SILENCE_MS = 5_000L
private var appContext: Context? = null private var appContext: Context? = null
/** True between onCreate and onDestroy; lets the activity tell a live capture from a crashed one. */
@Volatile var running = false
private set
fun start(context: Context, resultCode: Int, data: Intent) { fun start(context: Context, resultCode: Int, data: Intent) {
appContext = context.applicationContext appContext = context.applicationContext
context.startForegroundService( context.startForegroundService(
@@ -27,10 +27,16 @@ class AudioLink(
private var configIndex = 0 private var configIndex = 0
@Volatile private var ready = false @Volatile private var ready = false
@Volatile private var started = false @Volatile private var started = false
@Volatile private var focusGranted = false
@Volatile private var focusRequested = false
@Volatile private var inFlight = 0 @Volatile private var inFlight = 0
@Volatile private var maxUnacked = 8 // The Uconnect asks for 4 on its video channel; assume the same until its Config says otherwise.
@Volatile private var maxUnacked = 4
@Volatile private var lastAckAt = 0L @Volatile private var lastAckAt = 0L
private var chunks = 0 private var chunks = 0
private var bytes = 0L
private var dropped = 0
@Volatile private var configTimer: Thread? = null
private val config get() = service.configs.getOrNull(configIndex) ?: service.configs.firstOrNull() private val config get() = service.configs.getOrNull(configIndex) ?: service.configs.firstOrNull()
override val sampleRate get() = config?.sampleRate ?: 48_000 override val sampleRate get() = config?.sampleRate ?: 48_000
@@ -39,12 +45,47 @@ class AudioLink(
val channelId get() = service.serviceId val channelId get() = service.serviceId
fun opened(status: Int) { fun opened(status: Int) {
GearslipLog.i("audio: <- ChannelOpenResponse (media audio): status=$status")
if (status != 0) { if (status != 0) {
GearslipLog.w("audio: the head unit refused the media audio channel (status $status)") GearslipLog.w("audio: the head unit refused the media audio channel (status $status)")
return return
} }
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, service.codecType.toLong())) // AVChannelSetupRequest { required uint32 config_index = 1 }. This field is an index into
GearslipLog.i("audio: -> Setup(codec=${service.codecName})") // the sink's own audio_configs, not a codec id - sending the codec type asks for a config
// the sink does not have, and it answers with a status that never reaches READY.
sendOnChannel(MSG_SETUP, Protobuf.varintField(1, 0L))
GearslipLog.i("audio: -> Setup(config_index=0) on channel ${service.serviceId}, codec ${service.codecName}")
armConfigFallback()
}
/**
* The 2018 Uconnect answers our Setup with a two-byte message we cannot identify (id 255, no
* body) rather than the AVChannelSetupResponse the video channel gets, so the sink never
* reaches READY and no audio is ever sent.
*
* Service discovery already told us everything that message would have: the sink is PCM at
* 48000Hz stereo and it has exactly one configuration. So rather than wait forever for a
* message that is not coming, assume the advertised configuration and carry on. If that guess
* is wrong the head unit simply will not ack, which the write path already reports.
*/
private fun armConfigFallback() {
val timer = Thread({
try {
Thread.sleep(CONFIG_WAIT_MS)
} catch (e: InterruptedException) {
return@Thread
}
if (ready) return@Thread
GearslipLog.w(
"audio: no usable Config after ${CONFIG_WAIT_MS}ms - falling back to the sink the " +
"head unit advertised (${sampleRate}Hz x$channels, config_index=0, max_unacked=$maxUnacked)",
)
configIndex = 0
ready = true
CarAudio.attach(this)
}, "gearslip-audio-config").apply { isDaemon = true }
configTimer = timer
timer.start()
} }
fun onMessage(messageId: Int, body: ByteArray) { fun onMessage(messageId: Int, body: ByteArray) {
@@ -60,9 +101,15 @@ class AudioLink(
if (window != null) maxUnacked = maxOf(1, window) if (window != null) maxUnacked = maxOf(1, window)
configIndex = indices.firstOrNull() ?: 0 configIndex = indices.firstOrNull() ?: 0
if (status == STATUS_READY) { if (status == STATUS_READY) {
configTimer?.interrupt()
ready = true ready = true
GearslipLog.i("audio: ready - ${sampleRate}Hz x$channels ${config?.bits}bit") GearslipLog.i("audio: sink ready - ${sampleRate}Hz x$channels ${config?.bits}bit")
CarAudio.attach(this) CarAudio.attach(this)
} else {
GearslipLog.w(
"audio: the sink is not ready (status=$status, 1=WAIT 2=READY) - " +
"nothing will be sent until it is",
)
} }
} }
@@ -72,17 +119,40 @@ class AudioLink(
lastAckAt = System.currentTimeMillis() lastAckAt = System.currentTimeMillis()
} }
else -> GearslipLog.i("audio: <- unhandled message id=$messageId (${body.size} bytes)") // Whatever the head unit answers Setup with, we need its bytes to identify it.
else -> {
GearslipLog.i("audio: <- unhandled message id=$messageId (${body.size} bytes)")
GearslipLog.hex(" audio msg", body, limit = 64)
}
} }
} }
/**
* The head unit's answer to our AudioFocusRequest. Until it grants focus the sink is wired to
* whatever else the car is playing - the radio, usually - and anything we send is discarded,
* so [start] waits here rather than streaming into nothing.
*/
fun onFocus(state: Int) { fun onFocus(state: Int) {
GearslipLog.i("audio: <- AudioFocus state=$state (1=GAIN, 2=GAIN_TRANSIENT, 3=LOSS, 4=LOSS_TRANSIENT)") val name = FOCUS_STATES[state] ?: "unknown($state)"
GearslipLog.i("audio: <- AudioFocusResponse state=$name")
focusGranted = state == FOCUS_STATE_GAIN || state == FOCUS_STATE_GAIN_TRANSIENT
if (focusGranted) startStream() else stop()
} }
override fun start() { override fun start() {
if (!ready || started) return if (!ready || started) return
if (!focusRequested) {
focusRequested = true
sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_GAIN.toLong())) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_GAIN.toLong()))
GearslipLog.i("audio: -> AudioFocusRequest(GAIN); waiting for the head unit to hand the sink over")
}
// Some head units grant the sink silently and never answer. Do not let that be the end of
// it: start anyway, and let the acks say whether anything is being taken.
startStream()
}
private fun startStream() {
if (!ready || started) return
sendOnChannel( sendOnChannel(
MSG_START, MSG_START,
Protobuf.varintField(1, SESSION_ID.toLong()) + Protobuf.varintField(2, configIndex.toLong()), Protobuf.varintField(1, SESSION_ID.toLong()) + Protobuf.varintField(2, configIndex.toLong()),
@@ -90,7 +160,7 @@ class AudioLink(
inFlight = 0 inFlight = 0
lastAckAt = System.currentTimeMillis() lastAckAt = System.currentTimeMillis()
started = true started = true
GearslipLog.i("audio: -> AudioFocusRequest(GAIN), Start(config=$configIndex)") GearslipLog.i("audio: -> Start(session=$SESSION_ID, config=$configIndex) - streaming ${sampleRate}Hz x$channels")
} }
override fun stop() { override fun stop() {
@@ -98,17 +168,26 @@ class AudioLink(
started = false started = false
runCatching { runCatching {
sendOnChannel(MSG_STOP, Protobuf.varintField(1, SESSION_ID.toLong())) sendOnChannel(MSG_STOP, Protobuf.varintField(1, SESSION_ID.toLong()))
sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_RELEASE.toLong())) if (focusRequested) sendControl(MSG_FOCUS_REQUEST, Protobuf.varintField(1, FOCUS_RELEASE.toLong()))
} }
GearslipLog.i("audio: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks") focusRequested = false
focusGranted = false
GearslipLog.i("audio: -> Stop, AudioFocusRequest(RELEASE) after $chunks chunks / $bytes bytes")
chunks = 0 chunks = 0
bytes = 0
} }
override fun write(pcm: ByteArray, size: Int) { override fun write(pcm: ByteArray, size: Int) {
if (!started) return if (!started) return
if (inFlight >= maxUnacked) { if (inFlight >= maxUnacked) {
// A lost ack must not silence the stream for good; after a stall assume the window cleared. // A lost ack must not silence the stream for good; after a stall assume the window cleared.
if (System.currentTimeMillis() - lastAckAt > STALL_MS) inFlight = 0 else return if (System.currentTimeMillis() - lastAckAt > STALL_MS) {
GearslipLog.w("audio: no ack for ${STALL_MS}ms with $inFlight in flight - reopening the window")
inFlight = 0
} else {
dropped++
return
}
} }
val timestamp = ByteArray(8) val timestamp = ByteArray(8)
val micros = System.nanoTime() / 1_000 val micros = System.nanoTime() / 1_000
@@ -116,14 +195,26 @@ class AudioLink(
try { try {
sendOnChannel(MSG_DATA, timestamp + pcm.copyOf(size)) sendOnChannel(MSG_DATA, timestamp + pcm.copyOf(size))
} catch (e: java.io.IOException) { } catch (e: java.io.IOException) {
GearslipLog.e("audio: the link died mid-stream", e)
started = false started = false
return return
} }
inFlight++ inFlight++
chunks++ chunks++
bytes += size
// The first chunk proves the whole path; after that, an occasional line is enough to show
// the car is still taking audio and to catch a window that has quietly closed.
if (chunks == 1 || chunks % CHUNK_LOG_EVERY == 0) {
GearslipLog.i(
"audio: -> $chunks chunks / $bytes bytes, inFlight=$inFlight/$maxUnacked, " +
"dropped=$dropped, focus=${if (focusGranted) "granted" else "not granted"}",
)
}
} }
fun close() { fun close() {
configTimer?.interrupt()
configTimer = null
stop() stop()
ready = false ready = false
CarAudio.detach(this) CarAudio.detach(this)
@@ -142,7 +233,15 @@ class AudioLink(
private const val STATUS_READY = 2 private const val STATUS_READY = 2
private const val FOCUS_GAIN = 1 private const val FOCUS_GAIN = 1
private const val FOCUS_RELEASE = 4 private const val FOCUS_RELEASE = 4
private const val FOCUS_STATE_GAIN = 1
private const val FOCUS_STATE_GAIN_TRANSIENT = 2
private const val SESSION_ID = 2 private const val SESSION_ID = 2
private const val STALL_MS = 2_000L private const val STALL_MS = 2_000L
private const val CHUNK_LOG_EVERY = 200 // about every four seconds
private const val CONFIG_WAIT_MS = 2_500L
private val FOCUS_STATES = mapOf(
1 to "GAIN", 2 to "GAIN_TRANSIENT", 3 to "LOSS", 4 to "LOSS_TRANSIENT",
)
} }
} }
@@ -6,12 +6,15 @@ import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow import kotlinx.coroutines.flow.asStateFlow
/** /**
* Joins the two ends of the audio path: [AudioCaptureService] on the phone, which hears what a * Joins the two ends of the audio path: [AudioCaptureService] on the phone, which hears what the
* media app plays, and the car's audio channel, which is where it has to end up. * phone is playing, and the car's audio channel, which is where it has to end up.
* *
* They come and go independently. The phone side needs on-device consent and can be running on * Capture belongs to the car session, not to any one app. Android grants it through a consent
* the bench with no car attached; the car side exists only while a head unit is connected and * dialog that cannot be answered in advance or remembered, so Gearslip asks once, when a head unit
* offers a media audio sink. Whichever pair is present, samples flow. * first offers a sink to play through, and then holds that grant for as long as the car is
* connected. Switching between media apps, or starting playback from the phone, costs nothing
* further: the capture is by audio usage rather than by app, so whatever the phone plays as media
* goes to the car.
*/ */
object CarAudio { object CarAudio {
@@ -36,9 +39,8 @@ object CarAudio {
/** Set by the activity: shows the permission and consent dialogs, which only it can. */ /** Set by the activity: shows the permission and consent dialogs, which only it can. */
var onConsentRequested: (() -> Unit)? = null var onConsentRequested: (() -> Unit)? = null
/** The uid being captured (or waited on). */ /** Whether the driver has asked for the car to be fed at all; see CarSettings.pipeAudio. */
@Volatile var uid: Int = -1 @Volatile private var wanted = false
private set
@Volatile private var link: Link? = null @Volatile private var link: Link? = null
@@ -48,30 +50,48 @@ object CarAudio {
fun attach(link: Link) { fun attach(link: Link) {
this.link = link this.link = link
if (_capture.value == Capture.ON) link.start() GearslipLog.i("audio: the car offered a sink at ${link.sampleRate}Hz x${link.channels}")
when {
_capture.value == Capture.ON -> link.start()
wanted -> ask()
}
} }
fun detach(link: Link) { fun detach(link: Link) {
if (this.link === link) this.link = null if (this.link === link) this.link = null
} }
/** A media app was opened: start hearing it. */ /**
fun request(uid: Int) { * The car wants sound. Safe to call as often as the session likes: the consent dialog appears
if (this.uid == uid && _capture.value != Capture.OFF && _capture.value != Capture.DENIED) return * at most once, and only once there is a sink to play through - asking on the bench, where the
if (this.uid != uid) AudioCaptureService.stop() * sound has nowhere to go, would put a system dialog in front of the driver for nothing.
this.uid = uid */
_capture.value = Capture.ASKING fun request() {
val ask = onConsentRequested wanted = true
if (ask == null) { when (_capture.value) {
GearslipLog.w("audio: nothing on the phone can ask for capture consent") Capture.ON, Capture.ASKING, Capture.DENIED -> return
_capture.value = Capture.DENIED Capture.OFF -> if (link == null) {
GearslipLog.i("audio: waiting for the car to offer an audio sink")
} else { } else {
ask() ask()
} }
} }
}
private fun ask() {
_capture.value = Capture.ASKING
val request = onConsentRequested
if (request == null) {
GearslipLog.w("audio: nothing on the phone can ask for capture consent")
_capture.value = Capture.DENIED
} else {
request()
}
}
/** The car went away. Everything is torn down; the next connection asks again. */
fun release() { fun release() {
uid = -1 wanted = false
AudioCaptureService.stop() AudioCaptureService.stop()
_capture.value = Capture.OFF _capture.value = Capture.OFF
} }
@@ -0,0 +1,64 @@
package app.seb3thehacker.gearslip.audio
import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.provider.Settings
import app.seb3thehacker.gearslip.GearslipLog
/**
* Keeps the phone's notifications readable while audio is being sent to the car.
*
* Playback capture hangs off a MediaProjection, and since Android 15 any full-screen projection
* makes SystemUI hide notification contents, on the assumption that someone is watching the
* screen. Nobody is: the audio projection never makes a display. The one exemption an ordinary app
* can reach is the developer setting that switches the protection off, so Gearslip turns it off for
* the length of the capture and back on afterwards.
*
* Writing it needs WRITE_SECURE_SETTINGS, granted once over adb. Without the grant this does
* nothing and notifications stay hidden while music plays, which is the platform's default.
*
* Phone mirroring deliberately does not use this: there the screen really is being shown.
*/
internal object ScreenShareGuard {
/** Settings.Global.DISABLE_SCREEN_SHARE_PROTECTIONS_FOR_APPS_AND_NOTIFICATIONS, hidden API. */
private const val SETTING = "disable_screen_share_protections_for_apps_and_notifications"
private const val PREFS = "gearslip_screen_share_guard"
/** Set while Gearslip has the protection switched off, so a crash can be undone at next launch. */
private const val KEY_LIFTED = "lifted"
fun canLift(context: Context): Boolean =
context.checkSelfPermission(Manifest.permission.WRITE_SECURE_SETTINGS) ==
PackageManager.PERMISSION_GRANTED
/** Call before the projection starts: the system decides what to hide at that moment. */
fun lift(context: Context) {
if (!canLift(context)) {
GearslipLog.i("audio: notifications will be hidden while capturing (no WRITE_SECURE_SETTINGS)")
return
}
val resolver = context.contentResolver
// Already off, whether by the driver in Developer options or by us: leave it as found.
if (Settings.Global.getInt(resolver, SETTING, 0) != 0) return
runCatching {
prefs(context).edit().putBoolean(KEY_LIFTED, true).commit()
Settings.Global.putInt(resolver, SETTING, 1)
GearslipLog.i("audio: screen share protection off for the drive, notifications stay visible")
}.onFailure { GearslipLog.w("audio: could not lift screen share protection: ${it.message}") }
}
/** Put the protection back, if Gearslip is the one that took it away. */
fun restore(context: Context) {
if (!prefs(context).getBoolean(KEY_LIFTED, false)) return
runCatching {
Settings.Global.putInt(context.contentResolver, SETTING, 0)
prefs(context).edit().remove(KEY_LIFTED).commit()
GearslipLog.i("audio: screen share protection back on")
}.onFailure { GearslipLog.w("audio: could not restore screen share protection: ${it.message}") }
}
private fun prefs(context: Context) =
context.applicationContext.getSharedPreferences(PREFS, Context.MODE_PRIVATE)
}
@@ -62,11 +62,12 @@ object CarServices {
val live = phase == CarMedia.Phase.READY || phase == CarMedia.Phase.CONNECTING val live = phase == CarMedia.Phase.READY || phase == CarMedia.Phase.CONNECTING
if (_mediaApp.value?.component == app.component && live) return if (_mediaApp.value?.component == app.component && live) return
CarAudio.release()
_mediaApp.value = app _mediaApp.value = app
autoplayPending = autoplay autoplayPending = autoplay
media.connect(app) media.connect(app)
if (CarSettings.pipeAudio.value) CarAudio.request(app.uid) // Not torn down and rebuilt per app: the capture covers whatever the phone plays, so
// switching media apps never costs the driver another consent dialog.
if (CarSettings.pipeAudio.value) CarAudio.request()
} }
/** Called once the media app is connected: starts playback if that was asked for. */ /** Called once the media app is connected: starts playback if that was asked for. */
@@ -33,7 +33,7 @@ object CarSettings {
private val scaleFlow = MutableStateFlow(1f) private val scaleFlow = MutableStateFlow(1f)
private val openFlow = MutableStateFlow(HOME) private val openFlow = MutableStateFlow(HOME)
private val nightFlow = MutableStateFlow(NightMode.AUTO) private val nightFlow = MutableStateFlow(NightMode.AUTO)
private val pipeAudioFlow = MutableStateFlow(false) private val pipeAudioFlow = MutableStateFlow(true)
private val lastNavFlow = MutableStateFlow<String?>(null) private val lastNavFlow = MutableStateFlow<String?>(null)
private val lastMediaFlow = MutableStateFlow<String?>(null) private val lastMediaFlow = MutableStateFlow<String?>(null)
private val autoplayFlow = MutableStateFlow(false) private val autoplayFlow = MutableStateFlow(false)
@@ -52,9 +52,10 @@ object CarSettings {
val nightMode: StateFlow<NightMode> = nightFlow val nightMode: StateFlow<NightMode> = nightFlow
/** /**
* Whether a media app's sound is captured and sent over the car link. Off by default: the * Whether a media app's sound is captured and sent over the car link. On by default, since
* platform only allows that behind a screen-capture consent dialog, and without it the app * sound in the car is the point of the exercise; the platform allows the capture only behind a
* simply plays through the phone's normal output (e.g. Bluetooth to the car). * consent dialog, which [app.seb3thehacker.gearslip.audio.CarAudio] raises once the car has
* somewhere to play it. Turned off, the app plays through the phone's normal output instead.
*/ */
val pipeAudio: StateFlow<Boolean> = pipeAudioFlow val pipeAudio: StateFlow<Boolean> = pipeAudioFlow
@@ -68,7 +69,7 @@ object CarSettings {
AppTheme.valueOf(AppSettings.getString(app, KEY_THEME, AppTheme.PHONE.name)) AppTheme.valueOf(AppSettings.getString(app, KEY_THEME, AppTheme.PHONE.name))
}.getOrDefault(AppTheme.PHONE) }.getOrDefault(AppTheme.PHONE)
autoplayFlow.value = AppSettings.getString(app, KEY_AUTOPLAY, "false") == "true" autoplayFlow.value = AppSettings.getString(app, KEY_AUTOPLAY, "false") == "true"
pipeAudioFlow.value = AppSettings.getString(app, KEY_PIPE_AUDIO, "false") == "true" pipeAudioFlow.value = AppSettings.getString(app, KEY_PIPE_AUDIO, "true") == "true"
nightFlow.value = runCatching { nightFlow.value = runCatching {
NightMode.valueOf(AppSettings.getString(app, KEY_NIGHT, NightMode.AUTO.name)) NightMode.valueOf(AppSettings.getString(app, KEY_NIGHT, NightMode.AUTO.name))
}.getOrDefault(NightMode.AUTO) }.getOrDefault(NightMode.AUTO)
@@ -98,9 +98,10 @@ fun CarSettingsScreen() {
pipeAudio, pipeAudio,
) { CarSettings.setPipeAudio(it) } ) { CarSettings.setPipeAudio(it) }
Text( Text(
"Phone output needs no permission: the app plays as it normally would, e.g. over " + "Through Gearslip sends the sound over the USB link. Android asks for consent the " +
"Bluetooth to the car. Through Gearslip sends the sound over the car link, and " + "first time a media app plays in the car, and some apps refuse to be captured at " +
"Android asks for screen-capture consent each time.", "all. Phone output needs no permission: the app plays as it normally would, for " +
"example over Bluetooth to the car.",
style = MaterialTheme.typography.bodySmall, style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
@@ -2,7 +2,6 @@ package app.seb3thehacker.gearslip.ui
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.width import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
@@ -31,7 +30,11 @@ import androidx.compose.runtime.remember
import androidx.compose.runtime.setValue import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier import androidx.compose.ui.Modifier
import android.widget.Toast
import androidx.compose.ui.platform.LocalClipboardManager
import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.platform.LocalContext
import androidx.compose.ui.text.AnnotatedString
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.text.font.FontWeight
import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.dp
import androidx.lifecycle.compose.collectAsStateWithLifecycle import androidx.lifecycle.compose.collectAsStateWithLifecycle
@@ -178,8 +181,13 @@ private fun StatusCard(status: SessionStatus.Snapshot) {
private fun Context.appVersionName(): String = private fun Context.appVersionName(): String =
runCatching { packageManager.getPackageInfo(packageName, 0).versionName }.getOrNull() ?: "?" runCatching { packageManager.getPackageInfo(packageName, 0).versionName }.getOrNull() ?: "?"
/** One step of the setup list: a sentence that says what to do, and optionally how. */ /**
private class Step(val action: String, val detail: String? = null) * One step of the setup list: a sentence that says what to do, and optionally how.
*
* [command] is for the one step that cannot be done on the phone. It is shown as something to
* copy rather than to read, because a mistyped package name fails silently.
*/
private class Step(val action: String, val detail: String? = null, val command: String? = null)
private val SETUP_STEPS = listOf( private val SETUP_STEPS = listOf(
Step("Use a car from 2020 or earlier.", "Newer head units reject the certificate."), Step("Use a car from 2020 or earlier.", "Newer head units reject the certificate."),
@@ -189,20 +197,69 @@ private val SETUP_STEPS = listOf(
), ),
Step("Disable or uninstall the Android Auto app."), Step("Disable or uninstall the Android Auto app."),
Step("Load the certificate.", "Open Settings and choose Import certificate."), Step("Load the certificate.", "Open Settings and choose Import certificate."),
Step(
"Allow the microphone, so music can reach the car.",
"Android asks for the microphone before it will let any app pass music along. " +
"Gearslip never listens to you: it copies what a music app is playing, and nothing else.",
),
Step(
"Optional: stop Android asking about audio every drive.",
"Connect the phone to a computer with USB debugging turned on, and run this command. " +
"It tells Android to trust Gearslip with audio from now on. Skip it and everything " +
"still works, but you have to tap Start once each time you set off.",
command = "adb shell appops set app.seb3thehacker.gearslip PROJECT_MEDIA allow",
),
Step(
"Optional: keep notifications readable while music plays.",
"Android treats sending audio to the car like sharing your screen, and hides what your " +
"notifications say. Run this command too, and Gearslip switches that off only while " +
"music is going to the car, then puts it back.",
command = "adb shell pm grant app.seb3thehacker.gearslip android.permission.WRITE_SECURE_SETTINGS",
),
Step("Plug the phone into the car.", "Gearslip opens by itself when the head unit connects."), Step("Plug the phone into the car.", "Gearslip opens by itself when the head unit connects."),
) )
/** /**
* What to do before the first drive, in order. Kept in a narrow card with room at the sides so * A command to run on a computer, shown so it can be copied rather than retyped. Tapping it puts
* the lines stay short enough to read at a glance. * it on the clipboard, which is the only way it reaches the computer from here anyway.
*/ */
@Composable @Composable
private fun CopyableCommand(command: String) {
val clipboard = LocalClipboardManager.current
val context = LocalContext.current
Surface(
color = MaterialTheme.colorScheme.surfaceContainerHighest,
shape = MaterialTheme.shapes.small,
onClick = {
clipboard.setText(AnnotatedString(command))
Toast.makeText(context, "Command copied", Toast.LENGTH_SHORT).show()
},
modifier = Modifier.fillMaxWidth().padding(top = 4.dp),
) {
Column(Modifier.padding(horizontal = 12.dp, vertical = 10.dp), verticalArrangement = Arrangement.spacedBy(6.dp)) {
Text(
command,
style = MaterialTheme.typography.bodySmall,
fontFamily = FontFamily.Monospace,
color = MaterialTheme.colorScheme.onSurface,
)
Text(
"Tap to copy",
style = MaterialTheme.typography.labelSmall,
color = MaterialTheme.colorScheme.primary,
)
}
}
}
/** What to do before the first drive, in order. The same width as the status card above it. */
@Composable
private fun SetupSteps() { private fun SetupSteps() {
Box(Modifier.fillMaxWidth(), contentAlignment = Alignment.Center) { Box(Modifier.fillMaxWidth(), contentAlignment = Alignment.Center) {
Surface( Surface(
color = MaterialTheme.colorScheme.surfaceContainerLow, color = MaterialTheme.colorScheme.surfaceContainerLow,
shape = MaterialTheme.shapes.extraLarge, shape = MaterialTheme.shapes.extraLarge,
modifier = Modifier.widthIn(max = 560.dp).padding(horizontal = 16.dp), modifier = Modifier.fillMaxWidth(),
) { ) {
Column(Modifier.padding(horizontal = 24.dp, vertical = 22.dp), verticalArrangement = Arrangement.spacedBy(18.dp)) { Column(Modifier.padding(horizontal = 24.dp, vertical = 22.dp), verticalArrangement = Arrangement.spacedBy(18.dp)) {
Text( Text(
@@ -229,6 +286,7 @@ private fun SetupSteps() {
color = MaterialTheme.colorScheme.onSurfaceVariant, color = MaterialTheme.colorScheme.onSurfaceVariant,
) )
} }
step.command?.let { CopyableCommand(it) }
} }
} }
} }
@@ -0,0 +1,94 @@
package app.seb3thehacker.gearslip
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotNull
import org.junit.Test
/**
* Parses the ServiceDiscoveryResponse the 2018 Uconnect actually sent, rebuilt field for field
* from the decoded tree in `archive/spike/car-run-2026-09-16-run2.log`.
*
* The car offers two audio sinks - media on channel 5 and guidance on channel 6 - alongside the
* video sink on channel 1 and a microphone source on channel 3. Picking the right one out of that
* set is the whole job of [ServiceDiscovery.findAudioServices], and getting it wrong is silent:
* audio simply never arrives.
*/
class ServiceDiscoveryAudioTest {
private fun varint(value: Long): ByteArray = Protobuf.varint(value)
private fun varintField(field: Int, value: Long) = Protobuf.varintField(field, value)
private fun lenField(field: Int, body: ByteArray): ByteArray =
varint(((field shl 3) or 2).toLong()) + varint(body.size.toLong()) + body
/** AudioConfiguration { sampling_rate = 1; bits = 2; channels = 3 } */
private fun audioConfig(rate: Int, bits: Int, channels: Int) =
varintField(1, rate.toLong()) + varintField(2, bits.toLong()) + varintField(3, channels.toLong())
/** Service { id = 1; media_sink_service = 3 } carrying an audio sink. */
private fun audioSink(id: Int, codec: Int, audioType: Int, config: ByteArray) =
lenField(
1,
varintField(1, id.toLong()) +
lenField(3, varintField(1, codec.toLong()) + varintField(2, audioType.toLong()) + lenField(3, config)),
)
/** Channel 1: the H.264 video sink, which must not be mistaken for an audio one. */
private fun videoSink(): ByteArray {
val videoConfig = varintField(1, 2L) + varintField(2, 2L) + varintField(3, 0L) +
varintField(4, 12L) + varintField(5, 240L) + varintField(6, 0L) + varintField(8, 10_000L)
return lenField(
1,
varintField(1, 1L) + lenField(3, varintField(1, 3L) + lenField(4, videoConfig)),
)
}
/** Channel 3: the car's microphone, a media *source* (field 5), not a sink. */
private fun micSource(): ByteArray = lenField(
1,
varintField(1, 3L) + lenField(5, varintField(1, 1L) + lenField(2, audioConfig(16_000, 16, 1))),
)
private fun uconnectResponse(): ByteArray =
videoSink() +
micSource() +
audioSink(id = 5, codec = 1, audioType = 3, config = audioConfig(48_000, 16, 2)) +
audioSink(id = 6, codec = 1, audioType = 1, config = audioConfig(16_000, 16, 1)) +
Protobuf.stringField(2, "Uconnect") +
Protobuf.stringField(3, "Dodge Challenger") +
Protobuf.stringField(4, "18")
@Test
fun `finds both of the car's audio sinks and neither the video sink nor the microphone`() {
val sinks = ServiceDiscovery.findAudioServices(uconnectResponse())
assertEquals(listOf(5, 6), sinks.map { it.serviceId })
assertEquals(listOf("MEDIA", "GUIDANCE"), sinks.map { it.streamName })
}
@Test
fun `the media sink is PCM at 48kHz stereo`() {
val media = ServiceDiscovery.findAudioServices(uconnectResponse()).first { it.streamType == 3 }
assertEquals("AUDIO_PCM", media.codecName)
assertEquals(1, media.configs.size)
assertEquals(48_000, media.configs[0].sampleRate)
assertEquals(16, media.configs[0].bits)
assertEquals(2, media.configs[0].channels)
}
@Test
fun `the guidance sink is mono at 16kHz, so it must not be used for music`() {
val guidance = ServiceDiscovery.findAudioServices(uconnectResponse()).first { it.streamType == 1 }
assertEquals(16_000, guidance.configs[0].sampleRate)
assertEquals(1, guidance.configs[0].channels)
}
@Test
fun `the video sink is still found alongside the audio ones`() {
assertNotNull(ServiceDiscovery.findVideoService(uconnectResponse()))
assertEquals(1, ServiceDiscovery.findVideoService(uconnectResponse())!!.serviceId)
}
}