RetroArch/audio/audio_thread_wrapper.c
Jesse Talavera-Greenberg 938d60d0f4
Add microphone support via a new driver (#14731)
* Some slight fixes

* Update libretro.h

* Log calls to RETRO_ENVIRONMENT_GET_MICROPHONE_INTERFACE

* Finish proof-of-concept for mic support

- It works, but doesn't support floating-point audio yet
- It may need to be resampled, too

* Add macros that aren't available in SDL 2

* Comment out a variable definition for now

- For C89 compliance

* Add some comments for clarity

* Let ALSA tolerate a null new_rate

* Partial ALSA microphone support

- Not yet tested
- Mic is created and destroyed
- Mic can also be paused or unpaused
- Mic is paused or unpaused with the rest of the driver
- Microphone is not yet read

* Install error logging in the ALSA driver

- It defers to RARCH_ERR

* Free the ALSA microphone in alsa_free

* Fix an indent

* First draft of alsa_read_microphone

* Deinitialize SDL Audio in sdl_audio_free

* Save and restore the ALSA error logger

- You should always practice safe global state

* Add newlines to some RARCH_ERRs

* Add some logging

* Check for the mic being active via settings instead of via flags

* Adjusted a log entry to be less misleading

- A frequency of 0Hz looks weird to the uninformed
- In reality, it means the driver used the requested frequency

* Fix an incorrect format string

* Tidy up logging in alsa.c

* Rename audio_enable_microphone to audio_enable_input

* Rename microphone_device to audio_input_device

* Add audio_input_latency and audio_input_block_frames settings

* Add all mic-related settings to the options menu

* Adjust logging for alsa.c

- Log the ALSA library version
- Add errno details

* Refer to the microphone in logs by name

* Use %u instead of %d for some log items

* Add input_samples_buf

* Remove an inaccurate comment

* Change type of input_samples_buf

* Clean up audio_driver_flush_microphone_input

* Comment convert_float_to_s16

- It helped me understand what it's doing
- Turns out it'll work just fine on mono audio

* Don't use the resampler for mic input

* Fix crash in the ALSA driver when reading from a mic

* Update some logging messages

* ALSA support now works for mics

* Reuse some common functions in alsa.c

* Add alsa_thread_microphone_t

* Refactor alsa.c

- Introduce alsa_init_pcm to init any PCM that we're using
- Vastly simplifies the implementation of alsa_init and alsa_init_microphone
- Will be used for the read-based versions next

* Make ALSA logging a little more consistent

* Clean up the mic with alsa_free_microphone if alsa_init_microphone fails

* Remove an unused function

* Move some cleanup in alsa.c to a common function

* First crack at mic support for alsathread

- Refactor some duplicate code into functions
- Use functions introduced in alsa.c
- Create and destroy the mic

* Slight cleanups for clarity

* Implement alsa_thread_set/get_microphone_state

* More work on alsathread

- No more crashing, but the mic just returns silence

* Slight cleanups for clarity

* Add alsa_set_mic_enabled_internal

- For setting the state of a microphone while considering its current state

* Use alsa_set_mic_enabled_internal

* Log a little more info

* Log when the audio driver is started/stopped

* Move base microphone driver code into a new directory

- Add microphone_driver.c to Makefile.common
- Rename functions as needed

* Initialize and deinitialize the microphone driver

* Implement sdl_microphone.c

* Un-const an argument

- In case the driver context needs to do any locking

* Revise comments for microphone_driver.h

* Remove an unimplemented function

* Remove some functions from the mic driver

* Remove mic functions from audio_thread_wrapper

* Remove mic functions from sdl_audio

* Fix microphone_null

* Split the mic code for the alsa audio drivers into microphone drivers

* Fix an extra struct member

* Add a setting for the mic driver

* Add a command to reinitialize the microphone driver

* Rename mic-related settings

* Add DRIVER_MICROPHONE_MASK to DRIVERS_CMD_ALL

* Rename audio_enable_input to microphone_enable

* Remove some labels from qt_options

* Search for microphone_driver within find_driver_nonempty

* Clean up some mic driver code

* Pending mics now return silence

* Adjust some logging and comments

* Some cleanup in the microphone driver

* Invert a flag check

- Oops

* Fix a log message

* Fix the wrong flags being checked

* Slight refactor of wasapi_init_device

- Add a data_flow parameter
- Declare it in a header
- In preparation for WASAPI mic support

* Add some WASAPI macros for _IAudioCaptureClient

* Move some common WASAPI functions to audio/common/wasapi.c

- They'll be used by the mic and the audio drivers

* Add wasapi_log_hr

* Generalize mmdevice_list_new to look for capture devices, too

* Fix a function declaration

* Move driver-specific device_list_new functions into their respective files

* Clean up some declarations

* First draft of wasapi microphone driver

* Add wasapi_microphone_device_list_free

* Change function parameter names to be consistent with microphone_driver

* Partially implement wasapi_microphone_read

- Mostly copied from the audio driver so far
- It doesn't compile yet
- But it'll be beautiful when I'm done with it

* Refactor the mic driver's functions

- Rename get_mic_active to mic_alive
- Split set_mic_active into start_mic and stop_mic
- Refactor the SDL mic driver accordingly

* Edit some WASAPI functions for logging and clarity

* Implement more of the WASAPI mic driver

* Rename write_event to read_event

* Pass the WASAPI driver context to the various read functions

* Mostly implement the read function for the WASAPI mic driver

* Fix a crash in microphone_driver

- Forgot to move the position of the name of null_driver

* Reduce some logging in wasapi common functions

- Only log the chosen audio client format, not all attempted ones

* Add some macro wrappers for IAudioClient methods

* Update mic driver configuration

- Make the mic driver configurable in the menu
- Add config items for WASAPI-related options similar to the audio driver

* Fix a menu entry scrolling through audio devices instead of mic devices

* Add some utility functions

* Expose the new utility functions in wasapi.h

* Add extra logging in the WASAPI common functions

* Add sharemode_name

* Use _IAudioClient_Initialize macro in some places

* Pass channels to wasapi_init_client

- Remember, mics are in mono

* Use _IAudioClient_Initialize macro some more

* Forgot to pass channels in some places

* Add some utility functions

* Forgot an #include

* Add wasapi_select_device_format

* Simplify the format selection logic in wasapi_init_client_sh

* Unset the microphone in wasapi_microphone_close_mic

- Ought to prevent a potential segfault

* Simplify some logging

* Fix incorrect value being passed to _IAudioCaptureClient_ReleaseBuffer

* Remove some unneeded logging

* Add some values to hresult_name

* Polish up wasapi_select_device_format

- Test for formats manually when Windows can't
- Add some debug logging
- Check for channels

* Compute the fields of WAVEFORMATEXTENSIBLE correctly

- As per the doc's stated requirements

* Simplify logic for WASAPI client creation

* Fix a potential hang in wasapi_microphone_read_shared_buffered

* Stop the microphone if the driver is stopped

* Don't name the microphone event

* Ensure that wasapi_init_client reports the correct format and rate

* Implement exclusive microphone read access for WASAPI

* Add _IAudioCaptureClient_GetNextPacketSize macro

* Organize cases in hresult_name

* Clear some extra fields if wasapi_set_format is setting a Pcm format

* Adjust some logs

* Adjust some logs

* Remove unneeded local vars

* Add a log

* Update wasapi.c

* Update wasapi.c

* Fix shared-mode mic support in WASAPI producing broken input

- Turns out it had nothing to do with shared mode

* Reuse a common function

- Remove wasapi_microphone_read_shared_buffered
- Rename wasapi_microphone_read_exclusive to wasapi_microphone_read_buffered

* Remove some code I was using for test purposes

* Clarify some language

* Double the default shared-mode mic buffer length

* Split getting a device's name into a separate function, then use it

* Fix the ALSA mic drivers

- To comply with changes I previously made to the mic driver interface

* Remove unused synchronization primitives from the SDL microphone driver

* Add sdl_microphone_mic_use_float

* Document audio_driver_state_flags

- I needed to understand these to see if similar flags were required for the mic driver

* Remove an unused function in wasapi.c

* Add and document flags in microphone_driver.h

* Remove driver-specific mic start/stop functions

- The mic driver itself doesn't do much processing
- That honor goes to individual mics

* Remove some unused fields in microphone_driver.h

* Add CMD_EVENT_MICROPHONE_STOP/START

* Remove unused functions from microphone_null

* Change how the mic driver state is referenced in some places

* Simplify the SDL microphone driver

- The driver backend no longer keeps a reference to the mic (the frontend does that)
- Remove functions that are no longer needed
- Don't track paused state, just query the mic itself

* Simplify the WASAPI microphone driver

- Don't track the driver running state or the microphone handle, the frontend does that now
- Remove support for unbuffered input (hunterk suggested that it wasn't necessary)

* Make microphone_wasapi_sh_buffer_length a uint, not an int

- It won't be negative anymore
- 0 now represents the default value

* Make the microphone frontend more robust

- Improve documentation for how various functions should be implemented
- Closes all microphones before freeing the driver (so backends don't have to)
- Tracks the enabled state of each microphone, so backends don't have to (but they still can)

* Stop the mic driver in core_unload_game

* Ensure mic support is compatible with the revised menu code

* Move alsa.h into audio/common

* Remove RETRO_ENVIRONMENT_GET_MICROPHONE_ENABLED

- It was never really needed

* Refactor the ALSA microphone driver

- Move common ALSA functions to audio/common/alsa.c
- Replace alsa_set_mic_enabled_internal with alsa_start/stop_pcm
- Don't track the microphone handle in the ALSA driver context
- Remove unneeded fields

* Move some common alsathread code into audio/common/alsathread.c

* Change return type of mic_driver_open_mic_internal to bool

* First crack at resampling mic input

* Remove an extraneous check

- I think something distracted me when I was writing this line

* Add stereo/mono conversion functions

* Make alsa_start_pcm and alsa_stop_pcm more robust

- They now return success if the stream is already running and stopped, respectively

* Revise some mic-related comments in libretro.h

* First crack at resampling mic input

* Simplify an expression

* Simplify an expression

* Fix a log tag

* Allow mic resampler to be configured separately from audio resampler

* Add some comments

* Set the source ratio to something sensible

* Stop deadlock in `alsathread` mic driver

* Allow mics to be initialized even when core is loaded from CLI

- When loading content from CLI, the drivers are initialized a little differently
- That threw off the mic initialization code

* Rename the functions in retro_microphone_interface

* Revise some mic-related comments in libretro.h

* Update retro_microphone_interface

- Add get_mic_rate
- Add a parameter to open_mic
- The modifications don't do anything yet

* Use parameter objects in the microphone handle

* Replace get_mic_rate with get_params

* Add a microphone interface version

* Remove part of a comment

* Set the effective params in mic_driver_microphone_handle_init

* Drop a stray newline

* Change where the mic interface is zeroed

- I was accidentally throwing out the version that the core was asking for

* Reduce logspam for wasapi_set_nonblock_state

- Now it only logs when the sync mode is changed

* Change DEFAULT_WASAPI_SH_BUFFER_LENGTH to 0

- -16 is no longer a valid value

* Set the new_rate in wasapi_init

* Change description of microphone sample rate in the settings

* First attempt at resampling configured mic input

* Forgot a section

* Fix some input samples being skipped

* Rename a variable for clarity

* Add microphone.outgoing_samples

* Update the mic driver

- Processed samples are now buffered
- The resampler is skipped if the ratio is (very close to) 1

* Remove part of a comment

* Update some comments in audio_resampler.h

* Slightly refactor the SDL microphone driver

- Move SDL_AudioSpec to a field of sdl_microphone_handle_t
- Allow SDL to change the requested format and sample rate
- Request floating-point input
- Implement sdl_microphone_mic_use_float

* Fix a non-C89-compliant declaration

* Add new files to griffin.c

* Remove a C++-style comment

* Add two more files to griffin.c

* Remove some unneeded declarations in microphone_driver.h

* Remove a stray comma in configuration.c

- For C89 compliance

* Fix compilation on some platforms

* Change some function signatures

* Make the ALSA drivers always set the audio rate

* Fix the alsathread mic driver

* Make state_manager_frame_is_reversed return false if HAVE_REWIND isn't defined

* Mute the microphone if the core is running in fast-forward, slow-mo, or rewind

* Clarify a comment

* Clarify a comment

* Add a comment

* Don't allocate memory for slowmo samples in the mic driver

- We're not supporting slowmo for mics, so it's not needed

* Fix a {

* Add my name to AUTHORS.h

* Add driver_lifetime_flags

- For drivers that have special setup/teardown needs

* Ensure that resetting the mic driver maintains active mic handles

- Prevents fullscreen toggle from stopping all mic input

* Update CHANGES.md

* Move some default microphone settings to a new part of the config file

* Ensure that RetroArch can use the audio format that Windows suggests

* Remove references to mic support in the SDL audio driver

* Remove unused WASAPI functions

* Return failure if RetroArch couldn't select a WASAPI format

* Ensure that Windows uses the WASAPI mic driver by default

* Treat disabled mic support as a warning, not an error

* Clarify some WASAPI-related microphone settings

* Remove some unused variables

* Add or revise microphone-related comments

* Rearrange doc comments for microphone types in libretro.h

* Remove a space

* Remove some unused flags

* Remove ALSA error logger

- It was never used anyway

* Remove unneeded microphone-related arguments

* Document a parameter

* Remove a logging call

* Add a constant for the microphone's shared buffer length for WASAPI

* Fix stylistic inconsistencies

* Make mic_driver_get_sample_size a macro instead of a function

* Move the microphone implementation to the audio directory

* Make microphone support optional (but enabled by default)

* Fix the griffin build
2023-06-06 21:55:06 +02:00

351 lines
8.6 KiB
C

/* RetroArch - A frontend for libretro.
* Copyright (C) 2010-2014 - Hans-Kristian Arntzen
* Copyright (C) 2011-2017 - Daniel De Matteis
*
* RetroArch is free software: you can redistribute it and/or modify it under the terms
* of the GNU General Public License as published by the Free Software Found-
* ation, either version 3 of the License, or (at your option) any later version.
*
* RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
* without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
* PURPOSE. See the GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License along with RetroArch.
* If not, see <http://www.gnu.org/licenses/>.
*/
#include <stdlib.h>
#include <string.h>
#include <queues/fifo_queue.h>
#include <rthreads/rthreads.h>
#include "audio_thread_wrapper.h"
#include "audio_driver.h"
#include "../verbosity.h"
typedef struct audio_thread
{
const audio_driver_t *driver;
void *driver_data;
sthread_t *thread;
slock_t *lock;
scond_t *cond;
const char *device;
unsigned *new_rate;
int inited;
/* Initialization options. */
unsigned out_rate;
unsigned latency;
unsigned block_frames;
bool alive;
bool stopped;
bool stopped_ack;
bool is_paused;
bool is_shutdown;
bool use_float;
} audio_thread_t;
/**
* The thread that manages the life of the audio driver.
* The wrapped audio driver lives and dies with this function.
*/
static void audio_thread_loop(void *data)
{
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return;
thr->driver_data = thr->driver->init(
thr->device, thr->out_rate, thr->latency,
thr->block_frames, thr->new_rate);
slock_lock(thr->lock);
thr->inited = thr->driver_data ? 1 : -1;
if (thr->inited > 0 && thr->driver->use_float)
thr->use_float = thr->driver->use_float(thr->driver_data);
scond_signal(thr->cond);
slock_unlock(thr->lock);
if (thr->inited < 0)
return;
/* Wait until we start to avoid calling
* stop immediately after initialization. */
slock_lock(thr->lock);
while (thr->stopped)
scond_wait(thr->cond, thr->lock);
slock_unlock(thr->lock);
for (;;)
{
slock_lock(thr->lock);
if (!thr->alive)
{
scond_signal(thr->cond);
thr->stopped_ack = true;
slock_unlock(thr->lock);
break;
}
if (thr->stopped)
{
thr->driver->stop(thr->driver_data);
while (thr->stopped)
{
/* If we stop right after start,
* we might not be able to properly ack.
* Signal in the loop instead. */
thr->stopped_ack = true;
scond_signal(thr->cond);
scond_wait(thr->cond, thr->lock);
}
thr->driver->start(thr->driver_data, thr->is_shutdown);
}
slock_unlock(thr->lock);
audio_driver_callback();
}
thr->driver->free(thr->driver_data);
}
/**
* Lets the audio thread finish what it's doing,
* then stops it from doing further work.
*/
static void audio_thread_block(audio_thread_t *thr)
{
if (!thr)
return;
if (thr->stopped)
return;
slock_lock(thr->lock);
thr->stopped_ack = false;
thr->stopped = true;
scond_signal(thr->cond);
/* Wait until audio driver actually goes to sleep. */
while (!thr->stopped_ack)
scond_wait(thr->cond, thr->lock);
slock_unlock(thr->lock);
}
/**
* Resumes the audio thread.
* This function is called from the main thread.
*/
static void audio_thread_unblock(audio_thread_t *thr)
{
if (!thr)
return;
slock_lock(thr->lock); /* Prevent the audio thread from touching this flag... */
thr->stopped = false; /* ...so that the main thread can do it. */
scond_signal(thr->cond); /* Then let the audio thread know that it's okay to resume. */
slock_unlock(thr->lock); /* "As you were." */
}
static void audio_thread_free(void *data)
{
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return;
if (thr->thread)
{
slock_lock(thr->lock); /* Let the audio thread finish what it's doing... */
thr->stopped = false; /* Then stop it. "You're fired." */
thr->alive = false;
scond_signal(thr->cond); /* Let the thread know it's okay to continue */
slock_unlock(thr->lock); /* At this point, it will exit its loop. */
sthread_join(thr->thread);
/* Wait for the audio thread to exit, ensure that it's really dead.
* (It will call the wrapped driver's free() function.) */
}
if (thr->lock)
slock_free(thr->lock);
if (thr->cond)
scond_free(thr->cond);
free(thr);
/* The audio driver is done, clean up the thread itself. */
}
static bool audio_thread_alive(void *data)
{
bool alive = false;
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return false;
audio_thread_block(thr);
alive = !thr->is_paused;
audio_thread_unblock(thr);
return alive;
}
static bool audio_thread_stop(void *data)
{
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return false;
/* Don't immediately call stop on the driver;
* let the audio thread finish its current loop iteration.
* It will call stop then. */
audio_thread_block(thr);
thr->is_paused = true;
audio_driver_disable_callback();
return true;
}
static bool audio_thread_start(void *data, bool is_shutdown)
{
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return false;
audio_driver_enable_callback();
thr->is_paused = false;
thr->is_shutdown = is_shutdown;
audio_thread_unblock(thr);
return true;
}
static void audio_thread_set_nonblock_state(void *data, bool state)
{
(void)data;
(void)state;
/* Ignored, because blocking state is irrelevant
* when audio is running on a separate thread. */
}
static bool audio_thread_use_float(void *data)
{
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return false;
return thr->use_float;
}
static ssize_t audio_thread_write(void *data, const void *buf, size_t size)
{
ssize_t ret;
audio_thread_t *thr = (audio_thread_t*)data;
if (!thr)
return 0;
ret = thr->driver->write(thr->driver_data, buf, size);
if (ret < 0)
{
slock_lock(thr->lock);
thr->alive = false;
scond_signal(thr->cond);
slock_unlock(thr->lock);
}
return ret;
}
static const audio_driver_t audio_thread = {
NULL, /* No need to wrap init, it's called at the start of the thread loop */
audio_thread_write,
audio_thread_stop,
audio_thread_start,
audio_thread_alive,
audio_thread_set_nonblock_state,
audio_thread_free,
audio_thread_use_float,
"audio-thread",
NULL, /* No point in using rate control with threaded audio. */
NULL,
NULL,
NULL
};
/**
* audio_init_thread:
* @out_driver : output driver
* @out_data : output audio data
* @device : audio device (optional)
* @out_rate : output audio rate
* @latency : audio latency
* @driver : audio driver
*
* Starts a audio driver in a new thread.
* Access to audio driver will be mediated through this driver.
* This driver interfaces with audio callback and is
* only used in that case.
*
* Returns: true (1) if successful, otherwise false (0).
**/
bool audio_init_thread(const audio_driver_t **out_driver,
void **out_data, const char *device, unsigned audio_out_rate,
unsigned *new_rate, unsigned latency,
unsigned block_frames, const audio_driver_t *drv)
{
audio_thread_t *thr = (audio_thread_t*)calloc(1, sizeof(*thr));
if (!thr)
return false;
thr->driver = (const audio_driver_t*)drv;
thr->device = device;
thr->out_rate = audio_out_rate;
thr->new_rate = new_rate;
thr->latency = latency;
thr->block_frames = block_frames;
if (!(thr->cond = scond_new()))
goto error;
if (!(thr->lock = slock_new()))
goto error;
thr->alive = true;
thr->stopped = true;
if (!(thr->thread = sthread_create(audio_thread_loop, thr)))
goto error;
/* Wait until thread has initialized (or failed) the driver. */
slock_lock(thr->lock);
while (!thr->inited)
scond_wait(thr->cond, thr->lock);
slock_unlock(thr->lock);
if (thr->inited < 0) /* Thread failed. */
goto error;
*out_driver = &audio_thread;
*out_data = thr;
return true;
error:
*out_driver = NULL;
*out_data = NULL;
audio_thread_free(thr);
return false;
}