Jetpack Compose - Speech Recognition

android-jetpack-compose

Solution

I am going to explain my own implementation. Let me give you a general idea first, and then I am going to explain each step. So first you need to ask for permissions every time and then if permission is granted then you should start an intent in order to hear what the user says. What the user says is saved on a variable to a View Model. The variable on the View Model is being observed by the composable so you can get the data.

1) Add this to your Manifest file:

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    package="your.package">

    // Add uses-permission
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />

   [...]
   [...]
   [...]

    // Add above the last line  </manifest> like so:
    <queries>
        <intent>
            <action android:name="android.speech.RecognitionService" />
        </intent>
    </queries>

</manifest>

2) Create a ViewModel

class ScreenViewModel : ViewModel() {

    var textFromSpeech: String? by mutableStateOf(null)

}

You need the ViewModel in order to observe the variable from composable and implement your code logic for clean architecture.

3) Implement asking for permission

In `build.gradle` add the following:

implementation "com.google.accompanist:accompanist-permissions:$accompanist_version"

Then create a composable like so:

@ExperimentalPermissionsApi
@Composable
fun  OpenVoiceWithPermission(
    onDismiss: () -> Unit,
    vm: ScreenViewModel,
    ctxFromScreen: Context,
    finished: () -> Unit
) {

    val voicePermissionState = rememberPermissionState(android.Manifest.permission.RECORD_AUDIO)
    val ctx = LocalContext.current

fun newIntent(ctx: Context) {
    val intent = Intent()
    intent.action = Settings.ACTION_APPLICATION_DETAILS_SETTINGS
    val uri = Uri.fromParts(
        "package",
        BuildConfig.APPLICATION_ID, null
    )
    intent.data = uri
    intent.flags = Intent.FLAG_ACTIVITY_NEW_TASK
    ctx.startActivity(intent)
}

    PermissionRequired(
        permissionState = voicePermissionState,
        permissionNotGrantedContent = {
            DialogCustomBox(
                onDismiss = onDismiss,
                dialogBoxState = DialogLogInState.REQUEST_VOICE,
                onRequestPermission = { voicePermissionState.launchPermissionRequest() }
            )
        },
        permissionNotAvailableContent = {
            DialogCustomBox(
                onDismiss = onDismiss,
                dialogBoxState = DialogLogInState.VOICE_OPEN_SYSTEM_SETTINGS,
                onOpenSystemSettings = { newIntent(ctx) }
            )
        }
    ) {
        startSpeechToText(vm, ctxFromScreen, finished = finished)
    }
}

`DialogBox` you can create your own custom as I have done or use the standard version, this is up to you and out of the scope of this answer.

On the above code if permission is granted you move automatically to this piece of code: `startSpeechToText(vm, ctxFromScreen, finished = finished)` which you have to implement next.

4) Implementing Speech Recognizer

fun startSpeechToText(vm: ScreenViewModel, ctx: Context, finished: ()-> Unit) {
    val speechRecognizer = SpeechRecognizer.createSpeechRecognizer(ctx)
    val speechRecognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
    speechRecognizerIntent.putExtra(
        RecognizerIntent.EXTRA_LANGUAGE_MODEL,
        RecognizerIntent.LANGUAGE_MODEL_FREE_FORM,
    )

    // Optionally I have added my mother language
    speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, "el_GR")

    speechRecognizer.setRecognitionListener(object : RecognitionListener {
        override fun onReadyForSpeech(bundle: Bundle?) {}
        override fun onBeginningOfSpeech() {}
        override fun onRmsChanged(v: Float) {}
        override fun onBufferReceived(bytes: ByteArray?) {}
        override fun onEndOfSpeech() {
            finished()
            // changing the color of your mic icon to
            // gray to indicate it is not listening or do something you want
        }

        override fun onError(i: Int) {}

        override fun onResults(bundle: Bundle) {
            val result = bundle.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
            if (result != null) {
                // attaching the output
                // to our viewmodel
                vm.textFromSpeech = result[0]
            }
        }

        override fun onPartialResults(bundle: Bundle) {}
        override fun onEvent(i: Int, bundle: Bundle?) {}

    })
    speechRecognizer.startListening(speechRecognizerIntent)
}

With this implementation it is very customizable and you do not get this pop up from google. So you can inform the user that his device is listening with your own unique way!

5) Call from your composable the function to start listening:

@ExperimentalPermissionsApi
@Composable
fun YourScreen() {

    val ctx = LocalContext.current
    val vm: ScreenViewModel = viewModel()
    var clickToShowPermission by rememberSaveable { mutableStateOf(false) }

    if (clickToShowPermission) {
        OpenVoiceWithPermission(
            onDismiss = { clickToShowPermission = false },
            vm = vm,
            ctxFromScreen = ctx
        ) {
            // Do anything you want when the voice has finished and do
            // not forget to return clickToShowPermission to false!!
            clickToShowPermission = false
        }
    }
}

So on you code everytime you call `clickToShowPermission = true` you can start listening what the user says...

Problem

Original source