Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
ed75f06fb2 |
14 changed files with 792 additions and 8 deletions
15
CHANGELOG.md
15
CHANGELOG.md
|
|
@ -1,6 +1,19 @@
|
||||||
|
|
||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [1.0.8-rc33] - unstable
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
* [PostIt.Android] Avoid applying camera sensor rotation twice: account for the rotation already included in Camera2's SurfaceTexture matrix before rotating encoder input, while retaining portrait output dimensions.
|
||||||
|
* [PostIt.Android] Fix zero-second camera captures after adding OpenGL rotation: align Camera2 timestamps with the recorder's monotonic clock, preserve frame intervals, ignore duplicate frames and start camera requests only after the encoder is ready. Add a real Camera2/microphone regression check for segment duration and H.264/AAC tracks.
|
||||||
|
* [PostIt.Android] Keep the `VideoLAN.LibVLC.Android` JNI/native bindings alive in Release builds with a ProGuard rule set so the bundled `libvlc`/`libc++_shared` libraries are not stripped.
|
||||||
|
* [PostIt.Android] Rotate camera frames before H.264 encoding using an OpenGL surface, based on sensor orientation, lens facing and display rotation at capture start. Portrait captures swap the encoded dimensions; live MPEG-TS/HLS and replays no longer depend on an ineffective MP4-only orientation hint. Restore compilation by reading display orientation from the activity rather than the capture service.
|
||||||
|
|
||||||
## [1.0.8-rc32] - unstable
|
## [1.0.8-rc32] - unstable
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
@ -9,8 +22,8 @@
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
* [PostIt.Android] Upgrade `VideoLAN.LibVLC.Android` to 3.7.0-beta to resolve the runtime `failed to load libc++_shared.so` problem on 16 KiB-page Android devices and document the native-library compatibility requirement.
|
|
||||||
* Performer notification audience
|
* Performer notification audience
|
||||||
|
* [PostIt.Android] Upgrade `VideoLAN.LibVLC.Android` to 3.7.0-beta to resolve the runtime `failed to load libc++_shared.so` problem on 16 KiB-page Android devices and document the native-library compatibility requirement.
|
||||||
|
|
||||||
## [1.0.8-rc31] - unstable
|
## [1.0.8-rc31] - unstable
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -166,6 +166,15 @@ at IDR frames, with timestamps used for segment durations. Closed segments
|
||||||
enter the same ordered upload/retry/quota/replay pipeline as desktop.
|
enter the same ordered upload/retry/quota/replay pipeline as desktop.
|
||||||
Devices without a supported camera/encoder report a startup error.
|
Devices without a supported camera/encoder report a startup error.
|
||||||
|
|
||||||
|
Camera frames are rotated through an OpenGL surface before encoding, so both
|
||||||
|
live playback and replays have upright pixels without relying on rotation
|
||||||
|
metadata (not supported by the MPEG-TS recorder). The sensor, front/rear lens
|
||||||
|
and display orientation are sampled when capture starts; portrait output swaps
|
||||||
|
width and height. Camera2 may already include sensor rotation in the
|
||||||
|
SurfaceTexture transform; only the remaining rotation is applied by OpenGL.
|
||||||
|
Keep that device orientation during a broadcast, or stop and
|
||||||
|
start a new broadcast after turning the device.
|
||||||
|
|
||||||
A camera/microphone foreground service and a partial wake lock maintain the
|
A camera/microphone foreground service and a partial wake lock maintain the
|
||||||
broadcast when PostIt is backgrounded or the screen locks. Its ongoing
|
broadcast when PostIt is backgrounded or the screen locks. Its ongoing
|
||||||
notification returns to the app and offers **Arreter et publier**. Stopping
|
notification returns to the app and offers **Arreter et publier**. Stopping
|
||||||
|
|
|
||||||
|
|
@ -97,6 +97,59 @@ decodage authentifie du replay. Aucun FFmpeg n'est requis par ces tests ni
|
||||||
par l'application (le fichier d'entree peut provenir de n'importe quelle source).
|
par l'application (le fichier d'entree peut provenir de n'importe quelle source).
|
||||||
Les tests materiels camera/microphone Windows/Linux restent manuels.
|
Les tests materiels camera/microphone Windows/Linux restent manuels.
|
||||||
|
|
||||||
|
### Rotation de la capture Android
|
||||||
|
|
||||||
|
Le calcul capteur/ecran/camera avant et les quatre transformations de texture
|
||||||
|
sont couverts sans appareil :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet test --project src/PostIt/PostIt.Tests/PostIt.Tests.csproj \
|
||||||
|
--filter-class '*CameraFrameOrientationTests' --filter-class '*CameraFrameClockTests'
|
||||||
|
```
|
||||||
|
|
||||||
|
Un test Android supplementaire verifie le chemin OpenGL et l'encodeur reels,
|
||||||
|
sans ouvrir la camera, sans microphone et sans contacter l'API :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
dotnet build src/PostIt/PostIt.Android/PostIt.Android.csproj -c Debug \
|
||||||
|
-p:EmbedAssembliesIntoApk=true
|
||||||
|
adb install -r -t src/PostIt/PostIt.Android/bin/Debug/net10.0-android/fr.pschneider.postit-Signed.apk
|
||||||
|
adb shell am start -n fr.pschneider.postit/fr.pschneider.postit.CameraFrameRotationTestActivity
|
||||||
|
adb shell run-as fr.pschneider.postit cat files/camera-rotation-test-result.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Attendre la fin du test : `RUNNING` doit devenir `PASS`. Il encode quatre
|
||||||
|
quadrants colores en MPEG-TS pour 0, 90, 180 et 270 degres, decode une image et
|
||||||
|
verifie les dimensions (640x480 ou 480x640) et les couleurs aux quatre positions.
|
||||||
|
La tolerance de chaque composante RGB est de 40 pour tenir compte de H.264.
|
||||||
|
Une erreur de rendu, d'encodage, de decodage ou de comparaison produit `FAIL`
|
||||||
|
avec l'exception. Cette activite est exclue des builds Release.
|
||||||
|
|
||||||
|
Le test synthetique ne couvre pas l'horloge Camera2 ni sa synchronisation avec
|
||||||
|
le microphone. Avec l'accord de la personne utilisant le telephone, lancer
|
||||||
|
aussi le test local reel (camera et microphone actifs, aucun envoi a l'API) :
|
||||||
|
|
||||||
|
```bash
|
||||||
|
adb shell am start -n fr.pschneider.postit/fr.pschneider.postit.PodcastCaptureTestActivity --es microphone true
|
||||||
|
adb shell run-as fr.pschneider.postit cat files/podcast-capture-test-result.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
Attendre `PASS` : au moins trois segments, 8 a 16 secondes capturees,
|
||||||
|
pistes H.264 et AAC presentes, et progression des segments en arriere-plan.
|
||||||
|
Ce test detecte notamment les captures de zero seconde dues au decalage entre
|
||||||
|
l'horloge Camera2 et celle de MediaRecorder. Les fichiers restent dans le
|
||||||
|
repertoire prive indique par le resultat pour verification locale.
|
||||||
|
Le resultat inclut les angles capteur/ecran et la premiere matrice
|
||||||
|
SurfaceTexture recue. Les tests unitaires couvrent notamment la matrice
|
||||||
|
reelle `[0,-1,0,0,-1,0,0,0,0,0,1,0,1,1,0,1]` : pour un capteur a 90 degres
|
||||||
|
et un ecran a 0, elle porte deja la rotation et ne doit pas recevoir 90 degres
|
||||||
|
supplementaires. Le test synthetique seul ne detecte pas cette double rotation.
|
||||||
|
|
||||||
|
Verifier aussi manuellement les cameras avant/arriere en portrait et paysage,
|
||||||
|
le son, le direct et le replay, et la poursuite de l'emission en arriere-plan.
|
||||||
|
L'orientation est fixee au demarrage d'une emission ; tourner le telephone
|
||||||
|
pendant l'emission ne reconfigure pas le format encode.
|
||||||
|
|
||||||
## Quand ne PAS écrire de test
|
## Quand ne PAS écrire de test
|
||||||
|
|
||||||
Un test qui ne détecte rien n'est pas un test. Si l'invariant
|
Un test qui ne détecte rien n'est pas un test. Si l'invariant
|
||||||
|
|
|
||||||
|
|
@ -31,6 +31,7 @@
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
<ItemGroup Condition="'$(Configuration)' == 'Debug'">
|
<ItemGroup Condition="'$(Configuration)' == 'Debug'">
|
||||||
<Compile Include="..\PostIt.Tests\AndroidPodcastCaptureTestActivity.cs" Link="Tests\AndroidPodcastCaptureTestActivity.cs" />
|
<Compile Include="..\PostIt.Tests\AndroidPodcastCaptureTestActivity.cs" Link="Tests\AndroidPodcastCaptureTestActivity.cs" />
|
||||||
|
<Compile Include="..\PostIt.Tests\AndroidCameraFrameRotationTestActivity.cs" Link="Tests\AndroidCameraFrameRotationTestActivity.cs" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
<!-- This is an Avalonia app, not a MAUI app: exclude the .NET MAUI VS Code
|
<!-- This is an Avalonia app, not a MAUI app: exclude the .NET MAUI VS Code
|
||||||
extension's injected HotReload agent (references Microsoft.Maui.*).
|
extension's injected HotReload agent (references Microsoft.Maui.*).
|
||||||
|
|
|
||||||
308
src/PostIt/PostIt.Android/Services/AndroidCameraFrameRotator.cs
Normal file
308
src/PostIt/PostIt.Android/Services/AndroidCameraFrameRotator.cs
Normal file
|
|
@ -0,0 +1,308 @@
|
||||||
|
using System;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
using Android.Graphics;
|
||||||
|
using Android.Opengl;
|
||||||
|
using Android.OS;
|
||||||
|
using Android.Views;
|
||||||
|
using Java.Nio;
|
||||||
|
|
||||||
|
namespace PostIt.Android.Services;
|
||||||
|
|
||||||
|
internal sealed class AndroidCameraFrameRotator : IDisposable
|
||||||
|
{
|
||||||
|
private const int ExternalTexture = 0x8D65;
|
||||||
|
private const int RecordableAndroid = 0x3142;
|
||||||
|
private const string VertexShader = """
|
||||||
|
attribute vec2 position;
|
||||||
|
attribute vec2 textureCoordinate;
|
||||||
|
uniform mat4 textureMatrix;
|
||||||
|
varying vec2 uv;
|
||||||
|
void main() {
|
||||||
|
gl_Position = vec4(position, 0.0, 1.0);
|
||||||
|
uv = (textureMatrix * vec4(textureCoordinate, 0.0, 1.0)).xy;
|
||||||
|
}
|
||||||
|
""";
|
||||||
|
private const string FragmentShader = """
|
||||||
|
#extension GL_OES_EGL_image_external : require
|
||||||
|
precision mediump float;
|
||||||
|
uniform samplerExternalOES cameraTexture;
|
||||||
|
varying vec2 uv;
|
||||||
|
void main() {
|
||||||
|
gl_FragColor = texture2D(cameraTexture, uv);
|
||||||
|
}
|
||||||
|
""";
|
||||||
|
|
||||||
|
private readonly HandlerThread _thread = new("PostIt.CameraRotation");
|
||||||
|
private readonly Handler _handler;
|
||||||
|
private readonly Action<Exception> _failed;
|
||||||
|
private readonly float[] _textureMatrix = new float[16];
|
||||||
|
private readonly CameraFrameClock _clock = new();
|
||||||
|
private EGLDisplay? _display;
|
||||||
|
private EGLContext? _context;
|
||||||
|
private EGLSurface? _window;
|
||||||
|
private SurfaceTexture? _texture;
|
||||||
|
private FrameListener? _listener;
|
||||||
|
private Surface? _cameraSurface;
|
||||||
|
private FloatBuffer? _vertices;
|
||||||
|
private FloatBuffer? _coordinates;
|
||||||
|
private int _program;
|
||||||
|
private int _textureId;
|
||||||
|
private int _matrixLocation;
|
||||||
|
private bool _rendering;
|
||||||
|
private bool _disposed;
|
||||||
|
private int _rotation;
|
||||||
|
private int? _remainingRotation;
|
||||||
|
internal string? FirstTextureTransform { get; private set; }
|
||||||
|
|
||||||
|
internal Surface CameraSurface => _cameraSurface
|
||||||
|
?? throw new InvalidOperationException("Surface de rotation camera indisponible.");
|
||||||
|
|
||||||
|
private AndroidCameraFrameRotator(Action<Exception> failed)
|
||||||
|
{
|
||||||
|
_failed = failed;
|
||||||
|
_thread.Start();
|
||||||
|
_handler = new Handler(_thread.Looper!);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static async Task<AndroidCameraFrameRotator> CreateAsync(
|
||||||
|
Surface encoderSurface, int width, int height, int rotation, Action<Exception> failed)
|
||||||
|
{
|
||||||
|
var rotator = new AndroidCameraFrameRotator(failed);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await rotator.InvokeAsync(() => rotator.Initialize(encoderSurface, width, height, rotation));
|
||||||
|
return rotator;
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
rotator.Dispose();
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void Initialize(Surface encoderSurface, int width, int height, int rotation)
|
||||||
|
{
|
||||||
|
_rotation = rotation;
|
||||||
|
_display = EGL14.EglGetDisplay(EGL14.EglDefaultDisplay);
|
||||||
|
if (_display is null || _display.Equals(EGL14.EglNoDisplay))
|
||||||
|
throw EglError("Obtention de l'affichage EGL");
|
||||||
|
if (!EGL14.EglInitialize(_display, new int[2], 0, new int[2], 0))
|
||||||
|
throw EglError("Initialisation EGL");
|
||||||
|
int[] attributes =
|
||||||
|
[
|
||||||
|
EGL14.EglRedSize, 8, EGL14.EglGreenSize, 8, EGL14.EglBlueSize, 8,
|
||||||
|
EGL14.EglRenderableType, EGL14.EglOpenglEs2Bit,
|
||||||
|
EGL14.EglSurfaceType, EGL14.EglWindowBit, RecordableAndroid, 1, EGL14.EglNone
|
||||||
|
];
|
||||||
|
var configs = new EGLConfig[1];
|
||||||
|
var count = new int[1];
|
||||||
|
if (!EGL14.EglChooseConfig(_display, attributes, 0, configs, 0, 1, count, 0) || count[0] == 0)
|
||||||
|
throw EglError("Configuration EGL pour l'encodeur");
|
||||||
|
_context = EGL14.EglCreateContext(_display, configs[0], EGL14.EglNoContext,
|
||||||
|
[EGL14.EglContextClientVersion, 2, EGL14.EglNone], 0);
|
||||||
|
if (_context is null || _context.Equals(EGL14.EglNoContext))
|
||||||
|
throw EglError("Creation du contexte EGL");
|
||||||
|
_window = EGL14.EglCreateWindowSurface(_display, configs[0], encoderSurface, [EGL14.EglNone], 0);
|
||||||
|
if (_window is null || _window.Equals(EGL14.EglNoSurface))
|
||||||
|
throw EglError("Creation de la surface EGL");
|
||||||
|
if (!EGL14.EglMakeCurrent(_display, _window, _window, _context))
|
||||||
|
throw EglError("Activation du contexte EGL");
|
||||||
|
|
||||||
|
_program = CreateProgram();
|
||||||
|
var ids = new int[1];
|
||||||
|
GLES20.GlGenTextures(1, ids, 0);
|
||||||
|
_textureId = ids[0];
|
||||||
|
GLES20.GlBindTexture(ExternalTexture, _textureId);
|
||||||
|
GLES20.GlTexParameteri(ExternalTexture, GLES20.GlTextureMinFilter, GLES20.GlLinear);
|
||||||
|
GLES20.GlTexParameteri(ExternalTexture, GLES20.GlTextureMagFilter, GLES20.GlLinear);
|
||||||
|
GLES20.GlTexParameteri(ExternalTexture, GLES20.GlTextureWrapS, GLES20.GlClampToEdge);
|
||||||
|
GLES20.GlTexParameteri(ExternalTexture, GLES20.GlTextureWrapT, GLES20.GlClampToEdge);
|
||||||
|
_vertices = CreateBuffer([-1, -1, 1, -1, -1, 1, 1, 1]);
|
||||||
|
_coordinates = CreateBuffer(CameraFrameOrientation.GetTextureCoordinates(0));
|
||||||
|
GLES20.GlUseProgram(_program);
|
||||||
|
BindAttribute("position", _vertices);
|
||||||
|
BindAttribute("textureCoordinate", _coordinates);
|
||||||
|
_matrixLocation = GLES20.GlGetUniformLocation(_program, "textureMatrix");
|
||||||
|
GLES20.GlUniform1i(GLES20.GlGetUniformLocation(_program, "cameraTexture"), 0);
|
||||||
|
GLES20.GlViewport(0, 0, rotation is 90 or 270 ? height : width, rotation is 90 or 270 ? width : height);
|
||||||
|
CheckGl();
|
||||||
|
|
||||||
|
_texture = new SurfaceTexture(_textureId);
|
||||||
|
_texture.SetDefaultBufferSize(width, height);
|
||||||
|
_listener = new FrameListener(this);
|
||||||
|
_texture.SetOnFrameAvailableListener(_listener, _handler);
|
||||||
|
_cameraSurface = new Surface(_texture);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal Task StartAsync() => InvokeAsync(() => _rendering = true);
|
||||||
|
|
||||||
|
private void DrawFrame(SurfaceTexture texture)
|
||||||
|
{
|
||||||
|
if (!_rendering) return;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
texture.UpdateTexImage();
|
||||||
|
var presentationTime = _clock.GetPresentationTime(texture.Timestamp, Java.Lang.JavaSystem.NanoTime());
|
||||||
|
if (presentationTime is null) return;
|
||||||
|
texture.GetTransformMatrix(_textureMatrix);
|
||||||
|
var remainingRotation = CameraFrameOrientation.GetRemainingRotation(_rotation, _textureMatrix);
|
||||||
|
if (_remainingRotation != remainingRotation)
|
||||||
|
{
|
||||||
|
_remainingRotation = remainingRotation;
|
||||||
|
FirstTextureTransform = string.Join(", ", _textureMatrix);
|
||||||
|
_coordinates!.Position(0);
|
||||||
|
_coordinates.Put(CameraFrameOrientation.GetTextureCoordinates(remainingRotation));
|
||||||
|
_coordinates.Position(0);
|
||||||
|
global::Android.Util.Log.Info("PostIt.Podcast",
|
||||||
|
$"Camera texture transform: [{FirstTextureTransform}], remaining rotation={remainingRotation}");
|
||||||
|
}
|
||||||
|
GLES20.GlUniformMatrix4fv(_matrixLocation, 1, false, _textureMatrix, 0);
|
||||||
|
GLES20.GlDrawArrays(GLES20.GlTriangleStrip, 0, 4);
|
||||||
|
CheckGl();
|
||||||
|
if (!EGLExt.EglPresentationTimeANDROID(_display, _window, presentationTime.Value))
|
||||||
|
throw EglError("Horodatage de l'image camera");
|
||||||
|
if (!EGL14.EglSwapBuffers(_display, _window))
|
||||||
|
throw EglError("Envoi de l'image a l'encodeur");
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
_rendering = false;
|
||||||
|
_failed(new InvalidOperationException("Rotation de la capture Android impossible.", ex));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private Task InvokeAsync(Action action)
|
||||||
|
{
|
||||||
|
var completion = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
|
if (!_handler.Post(() =>
|
||||||
|
{
|
||||||
|
try { action(); completion.SetResult(); }
|
||||||
|
catch (Exception ex) { completion.SetException(ex); }
|
||||||
|
}))
|
||||||
|
completion.SetException(new InvalidOperationException("Le thread de rotation camera est arrete."));
|
||||||
|
return completion.Task;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static FloatBuffer CreateBuffer(float[] values)
|
||||||
|
{
|
||||||
|
using var bytes = ByteBuffer.AllocateDirect(values.Length * sizeof(float))
|
||||||
|
?? throw new InvalidOperationException("Allocation du tampon OpenGL impossible.");
|
||||||
|
bytes.Order(ByteOrder.NativeOrder()!);
|
||||||
|
var buffer = bytes.AsFloatBuffer()
|
||||||
|
?? throw new InvalidOperationException("Creation du tampon OpenGL impossible.");
|
||||||
|
buffer.Put(values);
|
||||||
|
buffer.Position(0);
|
||||||
|
return buffer;
|
||||||
|
}
|
||||||
|
|
||||||
|
private void BindAttribute(string name, FloatBuffer buffer)
|
||||||
|
{
|
||||||
|
var location = GLES20.GlGetAttribLocation(_program, name);
|
||||||
|
GLES20.GlEnableVertexAttribArray(location);
|
||||||
|
GLES20.GlVertexAttribPointer(location, 2, GLES20.GlFloat, false, 0, buffer);
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int CreateShader(int type, string source)
|
||||||
|
{
|
||||||
|
var shader = GLES20.GlCreateShader(type);
|
||||||
|
GLES20.GlShaderSource(shader, source);
|
||||||
|
GLES20.GlCompileShader(shader);
|
||||||
|
var status = new int[1];
|
||||||
|
GLES20.GlGetShaderiv(shader, GLES20.GlCompileStatus, status, 0);
|
||||||
|
if (status[0] != 0) return shader;
|
||||||
|
var error = GLES20.GlGetShaderInfoLog(shader);
|
||||||
|
GLES20.GlDeleteShader(shader);
|
||||||
|
throw new InvalidOperationException($"Compilation du shader camera impossible : {error}");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int CreateProgram()
|
||||||
|
{
|
||||||
|
var vertex = CreateShader(GLES20.GlVertexShader, VertexShader);
|
||||||
|
var fragment = 0;
|
||||||
|
var program = 0;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
fragment = CreateShader(GLES20.GlFragmentShader, FragmentShader);
|
||||||
|
program = GLES20.GlCreateProgram();
|
||||||
|
GLES20.GlAttachShader(program, vertex);
|
||||||
|
GLES20.GlAttachShader(program, fragment);
|
||||||
|
GLES20.GlLinkProgram(program);
|
||||||
|
var status = new int[1];
|
||||||
|
GLES20.GlGetProgramiv(program, GLES20.GlLinkStatus, status, 0);
|
||||||
|
if (status[0] == 0)
|
||||||
|
throw new InvalidOperationException($"Edition du programme camera impossible : {GLES20.GlGetProgramInfoLog(program)}");
|
||||||
|
return program;
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
if (program != 0) GLES20.GlDeleteProgram(program);
|
||||||
|
throw;
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
GLES20.GlDeleteShader(vertex);
|
||||||
|
if (fragment != 0) GLES20.GlDeleteShader(fragment);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void CheckGl()
|
||||||
|
{
|
||||||
|
var error = GLES20.GlGetError();
|
||||||
|
if (error != GLES20.GlNoError)
|
||||||
|
throw new InvalidOperationException($"Erreur OpenGL de capture : 0x{error:X}.");
|
||||||
|
}
|
||||||
|
|
||||||
|
private static InvalidOperationException EglError(string operation)
|
||||||
|
=> new($"{operation} impossible : EGL 0x{EGL14.EglGetError():X}.");
|
||||||
|
|
||||||
|
public void Dispose()
|
||||||
|
{
|
||||||
|
if (_disposed) return;
|
||||||
|
_disposed = true;
|
||||||
|
try { InvokeAsync(Release).GetAwaiter().GetResult(); }
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
_thread.QuitSafely();
|
||||||
|
_thread.Join();
|
||||||
|
_handler.Dispose();
|
||||||
|
_thread.Dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private void Release()
|
||||||
|
{
|
||||||
|
_rendering = false;
|
||||||
|
_texture?.SetOnFrameAvailableListener(null);
|
||||||
|
_cameraSurface?.Release();
|
||||||
|
_cameraSurface?.Dispose();
|
||||||
|
_texture?.Release();
|
||||||
|
_texture?.Dispose();
|
||||||
|
_listener?.Dispose();
|
||||||
|
if (_program != 0) GLES20.GlDeleteProgram(_program);
|
||||||
|
if (_textureId != 0) GLES20.GlDeleteTextures(1, [_textureId], 0);
|
||||||
|
_vertices?.Dispose();
|
||||||
|
_coordinates?.Dispose();
|
||||||
|
if (_display is not null && !_display.Equals(EGL14.EglNoDisplay))
|
||||||
|
{
|
||||||
|
EGL14.EglMakeCurrent(_display, EGL14.EglNoSurface, EGL14.EglNoSurface, EGL14.EglNoContext);
|
||||||
|
if (_window is not null && !_window.Equals(EGL14.EglNoSurface))
|
||||||
|
EGL14.EglDestroySurface(_display, _window);
|
||||||
|
if (_context is not null && !_context.Equals(EGL14.EglNoContext))
|
||||||
|
EGL14.EglDestroyContext(_display, _context);
|
||||||
|
EGL14.EglReleaseThread();
|
||||||
|
EGL14.EglTerminate(_display);
|
||||||
|
}
|
||||||
|
_window?.Dispose();
|
||||||
|
_context?.Dispose();
|
||||||
|
_display?.Dispose();
|
||||||
|
}
|
||||||
|
|
||||||
|
private sealed class FrameListener(AndroidCameraFrameRotator owner)
|
||||||
|
: Java.Lang.Object, SurfaceTexture.IOnFrameAvailableListener
|
||||||
|
{
|
||||||
|
public void OnFrameAvailable(SurfaceTexture? texture)
|
||||||
|
{
|
||||||
|
if (texture is not null) owner.DrawFrame(texture);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -26,6 +26,7 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
private CameraDevice? _camera;
|
private CameraDevice? _camera;
|
||||||
private CameraCaptureSession? _session;
|
private CameraCaptureSession? _session;
|
||||||
private MediaRecorder? _recorder;
|
private MediaRecorder? _recorder;
|
||||||
|
private AndroidCameraFrameRotator? _rotator;
|
||||||
private Surface? _surface;
|
private Surface? _surface;
|
||||||
private ParcelFileDescriptor? _readPipe;
|
private ParcelFileDescriptor? _readPipe;
|
||||||
private ParcelFileDescriptor? _writePipe;
|
private ParcelFileDescriptor? _writePipe;
|
||||||
|
|
@ -36,6 +37,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
private bool _started;
|
private bool _started;
|
||||||
private bool _disposed;
|
private bool _disposed;
|
||||||
public Task Completion => _completion.Task;
|
public Task Completion => _completion.Task;
|
||||||
|
internal string OrientationDiagnostics { get; private set; } = "";
|
||||||
|
internal string? TextureTransform => _rotator?.FirstTextureTransform;
|
||||||
|
|
||||||
private AndroidPodcastCapture(PodcastCaptureService service) => _service = service;
|
private AndroidPodcastCapture(PodcastCaptureService service) => _service = service;
|
||||||
|
|
||||||
|
|
@ -103,6 +106,30 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
var size = sizes.Where(s => s.Width <= 640 && s.Height <= 480)
|
var size = sizes.Where(s => s.Width <= 640 && s.Height <= 480)
|
||||||
.OrderByDescending(s => s.Width * s.Height).FirstOrDefault()
|
.OrderByDescending(s => s.Width * s.Height).FirstOrDefault()
|
||||||
?? throw new InvalidOperationException("La camera ne propose aucun format jusqu'a 480p.");
|
?? throw new InvalidOperationException("La camera ne propose aucun format jusqu'a 480p.");
|
||||||
|
using var sensor = characteristics.Get(CameraCharacteristics.SensorOrientation) as Java.Lang.Integer
|
||||||
|
?? throw new InvalidOperationException("Orientation du capteur camera indisponible.");
|
||||||
|
using var facing = characteristics.Get(CameraCharacteristics.LensFacing) as Java.Lang.Integer
|
||||||
|
?? throw new InvalidOperationException("Position du capteur camera indisponible.");
|
||||||
|
var activity = MainActivity.Current ?? throw new InvalidOperationException("Activite Android indisponible.");
|
||||||
|
#pragma warning disable CS0618, CA1422
|
||||||
|
var display = OperatingSystem.IsAndroidVersionAtLeast(30)
|
||||||
|
? activity.Display : activity.WindowManager?.DefaultDisplay;
|
||||||
|
#pragma warning restore CS0618, CA1422
|
||||||
|
var displayRotation = display?.Rotation
|
||||||
|
?? throw new InvalidOperationException("Orientation de l'ecran Android indisponible.");
|
||||||
|
var displayDegrees = displayRotation switch
|
||||||
|
{
|
||||||
|
SurfaceOrientation.Rotation0 => 0,
|
||||||
|
SurfaceOrientation.Rotation90 => 90,
|
||||||
|
SurfaceOrientation.Rotation180 => 180,
|
||||||
|
SurfaceOrientation.Rotation270 => 270,
|
||||||
|
_ => throw new InvalidOperationException($"Rotation d'ecran Android inconnue : {displayRotation}.")
|
||||||
|
};
|
||||||
|
var rotation = CameraFrameOrientation.GetRotation(sensor.IntValue(), displayDegrees,
|
||||||
|
facing.IntValue() == (int)LensFacing.Front);
|
||||||
|
global::Android.Util.Log.Info("PostIt.Podcast",
|
||||||
|
$"Camera orientation: sensor={sensor.IntValue()}, display={displayDegrees}, facing={facing.IntValue()}, rotation={rotation}");
|
||||||
|
OrientationDiagnostics = $"sensor={sensor.IntValue()}, display={displayDegrees}, facing={facing.IntValue()}, rotation={rotation}";
|
||||||
#pragma warning disable CS0618
|
#pragma warning disable CS0618
|
||||||
_recorder = OperatingSystem.IsAndroidVersionAtLeast(31)
|
_recorder = OperatingSystem.IsAndroidVersionAtLeast(31)
|
||||||
? new MediaRecorder(_service) : new MediaRecorder();
|
? new MediaRecorder(_service) : new MediaRecorder();
|
||||||
|
|
@ -111,7 +138,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
if (microphone) _recorder.SetAudioSource(AudioSource.Mic);
|
if (microphone) _recorder.SetAudioSource(AudioSource.Mic);
|
||||||
_recorder.SetVideoSource(VideoSource.Surface);
|
_recorder.SetVideoSource(VideoSource.Surface);
|
||||||
_recorder.SetOutputFormat(OutputFormat.Mpeg2Ts);
|
_recorder.SetOutputFormat(OutputFormat.Mpeg2Ts);
|
||||||
_recorder.SetVideoSize(size.Width, size.Height);
|
_recorder.SetVideoSize(rotation is 90 or 270 ? size.Height : size.Width,
|
||||||
|
rotation is 90 or 270 ? size.Width : size.Height);
|
||||||
_recorder.SetVideoFrameRate(30);
|
_recorder.SetVideoFrameRate(30);
|
||||||
_recorder.SetVideoEncodingBitRate(1200000);
|
_recorder.SetVideoEncodingBitRate(1200000);
|
||||||
_recorder.SetVideoEncoder(VideoEncoder.H264);
|
_recorder.SetVideoEncoder(VideoEncoder.H264);
|
||||||
|
|
@ -131,20 +159,23 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
ct.ThrowIfCancellationRequested();
|
ct.ThrowIfCancellationRequested();
|
||||||
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
|
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
|
||||||
_surface = _recorder.Surface ?? throw new InvalidOperationException("Surface encodeur indisponible.");
|
_surface = _recorder.Surface ?? throw new InvalidOperationException("Surface encodeur indisponible.");
|
||||||
|
_rotator = await AndroidCameraFrameRotator.CreateAsync(_surface, size.Width, size.Height, rotation, Fail);
|
||||||
|
ct.ThrowIfCancellationRequested();
|
||||||
|
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
|
||||||
|
var cameraSurface = _rotator.CameraSurface;
|
||||||
var opened = new TaskCompletionSource<CameraDevice>(TaskCreationOptions.RunContinuationsAsynchronously);
|
var opened = new TaskCompletionSource<CameraDevice>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
manager.OpenCamera(cameraId, new DeviceCallback(this, opened), _handler);
|
manager.OpenCamera(cameraId, new DeviceCallback(this, opened), _handler);
|
||||||
_camera = await opened.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
|
_camera = await opened.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
|
||||||
var configured = new TaskCompletionSource<CameraCaptureSession>(TaskCreationOptions.RunContinuationsAsynchronously);
|
var configured = new TaskCompletionSource<CameraCaptureSession>(TaskCreationOptions.RunContinuationsAsynchronously);
|
||||||
#pragma warning disable CS0618, CA1422
|
#pragma warning disable CS0618, CA1422
|
||||||
_camera.CreateCaptureSession(new List<Surface> { _surface }, new SessionCallback(this, configured), _handler);
|
_camera.CreateCaptureSession(new List<Surface> { cameraSurface }, new SessionCallback(this, configured), _handler);
|
||||||
#pragma warning restore CS0618, CA1422
|
#pragma warning restore CS0618, CA1422
|
||||||
_session = await configured.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
|
_session = await configured.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
|
||||||
ct.ThrowIfCancellationRequested();
|
ct.ThrowIfCancellationRequested();
|
||||||
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
|
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
|
||||||
using var request = _camera.CreateCaptureRequest(CameraTemplate.Record);
|
using var request = _camera.CreateCaptureRequest(CameraTemplate.Record);
|
||||||
request.AddTarget(_surface);
|
request.AddTarget(cameraSurface);
|
||||||
using var built = request.Build();
|
using var built = request.Build();
|
||||||
_session.SetRepeatingRequest(built, null, _handler);
|
|
||||||
var handle = new SafeFileHandle((IntPtr)_readPipe.Fd, ownsHandle: false);
|
var handle = new SafeFileHandle((IntPtr)_readPipe.Fd, ownsHandle: false);
|
||||||
var stream = new FileStream(handle, FileAccess.Read);
|
var stream = new FileStream(handle, FileAccess.Read);
|
||||||
_pump = Task.Run(async () =>
|
_pump = Task.Run(async () =>
|
||||||
|
|
@ -169,6 +200,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
});
|
});
|
||||||
await Task.Run(_recorder.Start, ct);
|
await Task.Run(_recorder.Start, ct);
|
||||||
_started = true;
|
_started = true;
|
||||||
|
await _rotator.StartAsync();
|
||||||
|
_session.SetRepeatingRequest(built, null, _handler);
|
||||||
_writePipe.Close();
|
_writePipe.Close();
|
||||||
_writePipe.Dispose();
|
_writePipe.Dispose();
|
||||||
_writePipe = null;
|
_writePipe = null;
|
||||||
|
|
@ -261,7 +294,11 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
|
|
||||||
private void StopNative()
|
private void StopNative()
|
||||||
{
|
{
|
||||||
try { if (_started) _recorder?.Stop(); }
|
try
|
||||||
|
{
|
||||||
|
// Keep frames flowing until MediaRecorder has drained its encoder.
|
||||||
|
if (_started) _recorder?.Stop();
|
||||||
|
}
|
||||||
catch (Java.Lang.RuntimeException ex)
|
catch (Java.Lang.RuntimeException ex)
|
||||||
{
|
{
|
||||||
_failure = ex;
|
_failure = ex;
|
||||||
|
|
@ -276,6 +313,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
|
||||||
_camera?.Close();
|
_camera?.Close();
|
||||||
_camera?.Dispose();
|
_camera?.Dispose();
|
||||||
_camera = null;
|
_camera = null;
|
||||||
|
_rotator?.Dispose();
|
||||||
|
_rotator = null;
|
||||||
if (_recorder is not null)
|
if (_recorder is not null)
|
||||||
{
|
{
|
||||||
_recorder.Error -= OnRecorderError;
|
_recorder.Error -= OnRecorderError;
|
||||||
|
|
|
||||||
24
src/PostIt/PostIt.Android/Services/CameraFrameClock.cs
Normal file
24
src/PostIt/PostIt.Android/Services/CameraFrameClock.cs
Normal file
|
|
@ -0,0 +1,24 @@
|
||||||
|
using System;
|
||||||
|
|
||||||
|
namespace PostIt.Android.Services;
|
||||||
|
|
||||||
|
internal sealed class CameraFrameClock
|
||||||
|
{
|
||||||
|
private long? _offset;
|
||||||
|
private long? _previousTimestamp;
|
||||||
|
|
||||||
|
internal long? GetPresentationTime(long cameraTimestamp, long monotonicNow)
|
||||||
|
{
|
||||||
|
if (_previousTimestamp is { } previous)
|
||||||
|
{
|
||||||
|
if (cameraTimestamp == previous) return null;
|
||||||
|
if (cameraTimestamp < previous)
|
||||||
|
throw new InvalidOperationException("Les horodatages de la camera Android ont recule.");
|
||||||
|
}
|
||||||
|
// Camera2 may use elapsed realtime (including suspend), whereas MediaRecorder
|
||||||
|
// and its microphone use monotonic time. Preserve frame intervals, not that epoch.
|
||||||
|
_offset ??= monotonicNow - cameraTimestamp;
|
||||||
|
_previousTimestamp = cameraTimestamp;
|
||||||
|
return checked(cameraTimestamp + _offset.Value);
|
||||||
|
}
|
||||||
|
}
|
||||||
55
src/PostIt/PostIt.Android/Services/CameraFrameOrientation.cs
Normal file
55
src/PostIt/PostIt.Android/Services/CameraFrameOrientation.cs
Normal file
|
|
@ -0,0 +1,55 @@
|
||||||
|
using System;
|
||||||
|
|
||||||
|
namespace PostIt.Android.Services;
|
||||||
|
|
||||||
|
internal static class CameraFrameOrientation
|
||||||
|
{
|
||||||
|
internal static int GetRotation(int sensorDegrees, int displayDegrees, bool frontFacing)
|
||||||
|
{
|
||||||
|
Validate(sensorDegrees);
|
||||||
|
Validate(displayDegrees);
|
||||||
|
return (sensorDegrees + (frontFacing ? displayDegrees : -displayDegrees) + 360) % 360;
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static float[] GetTextureCoordinates(int clockwiseDegrees)
|
||||||
|
{
|
||||||
|
Validate(clockwiseDegrees);
|
||||||
|
float[] corners = [0, 0, 1, 0, 0, 1, 1, 1];
|
||||||
|
for (var i = 0; i < corners.Length; i += 2)
|
||||||
|
{
|
||||||
|
var x = corners[i];
|
||||||
|
var y = corners[i + 1];
|
||||||
|
(corners[i], corners[i + 1]) = clockwiseDegrees switch
|
||||||
|
{
|
||||||
|
90 => (1 - y, x),
|
||||||
|
180 => (1 - x, 1 - y),
|
||||||
|
270 => (y, 1 - x),
|
||||||
|
_ => (x, y)
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return corners;
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static int GetRemainingRotation(int clockwiseDegrees, float[] textureMatrix)
|
||||||
|
{
|
||||||
|
Validate(clockwiseDegrees);
|
||||||
|
if (textureMatrix.Length != 16)
|
||||||
|
throw new ArgumentException("A SurfaceTexture transform must contain 16 elements.", nameof(textureMatrix));
|
||||||
|
// SurfaceTexture's usual Y flip changes the sign of the first column's Y.
|
||||||
|
// Camera2 can also put the sensor rotation in this matrix; don't apply it twice.
|
||||||
|
var angle = Math.Atan2(-textureMatrix[1], textureMatrix[0]) * 180 / Math.PI;
|
||||||
|
if (!double.IsFinite(angle) || (textureMatrix[0] == 0 && textureMatrix[1] == 0))
|
||||||
|
throw new ArgumentException("Invalid SurfaceTexture orientation.", nameof(textureMatrix));
|
||||||
|
var quarterTurns = (int)Math.Round(angle / 90);
|
||||||
|
if (Math.Abs(angle - quarterTurns * 90) > 0.01)
|
||||||
|
throw new ArgumentException("SurfaceTexture orientation is not a quarter turn.", nameof(textureMatrix));
|
||||||
|
var producerRotation = (quarterTurns * 90 + 360) % 360;
|
||||||
|
return (clockwiseDegrees - producerRotation + 360) % 360;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static void Validate(int degrees)
|
||||||
|
{
|
||||||
|
if (degrees is not (0 or 90 or 180 or 270))
|
||||||
|
throw new ArgumentOutOfRangeException(nameof(degrees), degrees, "Rotation must be 0, 90, 180 or 270 degrees.");
|
||||||
|
}
|
||||||
|
}
|
||||||
9
src/PostIt/PostIt.Android/proguard-rules.pro
vendored
Normal file
9
src/PostIt/PostIt.Android/proguard-rules.pro
vendored
Normal file
|
|
@ -0,0 +1,9 @@
|
||||||
|
# Keep the LibVLC Android classes and JNI/native bindings intact in Release builds.
|
||||||
|
# R8/ProGuard otherwise can strip or rename the Java-side entry points that load
|
||||||
|
# the native libvlc and libc++_shared libraries bundled by VideoLAN.LibVLC.Android.
|
||||||
|
-keep class org.videolan.libvlc.** { *; }
|
||||||
|
-keep class org.videolan.libvlc.** { native *; }
|
||||||
|
-keep class org.videolan.medialibrary.** { *; }
|
||||||
|
-keepattributes *Annotation*
|
||||||
|
-dontwarn org.videolan.libvlc.**
|
||||||
|
-dontwarn org.videolan.medialibrary.**
|
||||||
|
|
@ -0,0 +1,147 @@
|
||||||
|
#if ANDROID
|
||||||
|
using System;
|
||||||
|
using System.IO;
|
||||||
|
using System.Runtime.Versioning;
|
||||||
|
using System.Threading;
|
||||||
|
using System.Threading.Tasks;
|
||||||
|
using Android.App;
|
||||||
|
using Android.Graphics;
|
||||||
|
using Android.Media;
|
||||||
|
using Android.OS;
|
||||||
|
using PostIt.Android.Services;
|
||||||
|
using Path = System.IO.Path;
|
||||||
|
|
||||||
|
namespace PostIt.Tests;
|
||||||
|
|
||||||
|
/// <summary>Debug-only synthetic encoder test; never opens the camera or contacts the API.</summary>
|
||||||
|
[Activity(Name = "fr.pschneider.postit.CameraFrameRotationTestActivity", Exported = true)]
|
||||||
|
[SupportedOSPlatform("android26.0")]
|
||||||
|
public sealed class AndroidCameraFrameRotationTestActivity : Activity
|
||||||
|
{
|
||||||
|
protected override async void OnCreate(Bundle? state)
|
||||||
|
{
|
||||||
|
base.OnCreate(state);
|
||||||
|
var resultPath = Path.Combine(FilesDir!.AbsolutePath, "camera-rotation-test-result.txt");
|
||||||
|
File.WriteAllText(resultPath, "RUNNING");
|
||||||
|
try
|
||||||
|
{
|
||||||
|
foreach (var rotation in new[] { 0, 90, 180, 270 })
|
||||||
|
await VerifyRotationAsync(rotation);
|
||||||
|
File.WriteAllText(resultPath, "PASS\nrotations=0,90,180,270\nformat=MPEG-TS\nsource=synthetic");
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
File.WriteAllText(resultPath, $"FAIL\n{ex}");
|
||||||
|
}
|
||||||
|
finally { Finish(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
private async Task VerifyRotationAsync(int rotation)
|
||||||
|
{
|
||||||
|
const int width = 640;
|
||||||
|
const int height = 480;
|
||||||
|
var outputWidth = rotation is 90 or 270 ? height : width;
|
||||||
|
var outputHeight = rotation is 90 or 270 ? width : height;
|
||||||
|
var path = Path.Combine(FilesDir!.AbsolutePath, $"camera-rotation-{rotation}.ts");
|
||||||
|
void Stage(string stage) => File.WriteAllText(
|
||||||
|
Path.Combine(FilesDir.AbsolutePath, "camera-rotation-test-result.txt"), $"RUNNING\nrotation={rotation}\nstage={stage}");
|
||||||
|
Stage("prepare");
|
||||||
|
#pragma warning disable CS0618
|
||||||
|
using var recorder = OperatingSystem.IsAndroidVersionAtLeast(31)
|
||||||
|
? new MediaRecorder(this) : new MediaRecorder();
|
||||||
|
#pragma warning restore CS0618
|
||||||
|
recorder.SetVideoSource(VideoSource.Surface);
|
||||||
|
recorder.SetOutputFormat(OutputFormat.Mpeg2Ts);
|
||||||
|
recorder.SetVideoSize(outputWidth, outputHeight);
|
||||||
|
recorder.SetVideoFrameRate(30);
|
||||||
|
recorder.SetVideoEncodingBitRate(1200000);
|
||||||
|
recorder.SetVideoEncoder(VideoEncoder.H264);
|
||||||
|
recorder.SetOutputFile(path);
|
||||||
|
recorder.Prepare();
|
||||||
|
using var encoderSurface = recorder.Surface
|
||||||
|
?? throw new InvalidOperationException("Encoder surface unavailable.");
|
||||||
|
Exception? failure = null;
|
||||||
|
var stopping = 0;
|
||||||
|
Stage("initialize EGL");
|
||||||
|
using (var rotator = await AndroidCameraFrameRotator.CreateAsync(
|
||||||
|
encoderSurface, width, height, rotation, error =>
|
||||||
|
{
|
||||||
|
if (Volatile.Read(ref stopping) != 0) return;
|
||||||
|
failure = error;
|
||||||
|
File.WriteAllText(Path.Combine(FilesDir.AbsolutePath, "camera-rotation-test-result.txt"), $"FAIL\n{error}");
|
||||||
|
}))
|
||||||
|
{
|
||||||
|
Stage("start recorder");
|
||||||
|
recorder.Start();
|
||||||
|
Task? stop = null;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await rotator.StartAsync();
|
||||||
|
using var paint = new Paint();
|
||||||
|
Color[] colors = [Color.Red, Color.Green, Color.Blue, Color.Yellow];
|
||||||
|
Stage("draw frames");
|
||||||
|
for (var frame = 0; frame < 90; frame++)
|
||||||
|
{
|
||||||
|
if (failure is not null) throw failure;
|
||||||
|
if (frame == 45)
|
||||||
|
{
|
||||||
|
Volatile.Write(ref stopping, 1);
|
||||||
|
stop = Task.Run(recorder.Stop);
|
||||||
|
}
|
||||||
|
if (stop?.IsCompleted == true) break;
|
||||||
|
var canvas = rotator.CameraSurface.LockCanvas(null)
|
||||||
|
?? throw new InvalidOperationException("Synthetic camera canvas unavailable.");
|
||||||
|
try
|
||||||
|
{
|
||||||
|
for (var i = 0; i < colors.Length; i++)
|
||||||
|
{
|
||||||
|
paint.Color = colors[i];
|
||||||
|
var x = i % 2 * width / 2;
|
||||||
|
var y = i / 2 * height / 2;
|
||||||
|
canvas.DrawRect(x, y, x + width / 2, y + height / 2, paint);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
rotator.CameraSurface.UnlockCanvasAndPost(canvas);
|
||||||
|
canvas.Dispose();
|
||||||
|
}
|
||||||
|
await Task.Delay(34);
|
||||||
|
}
|
||||||
|
if (failure is not null) throw failure;
|
||||||
|
Stage("wait for recorder stop");
|
||||||
|
await (stop ?? throw new InvalidOperationException("Recorder stop was not requested."))
|
||||||
|
.WaitAsync(TimeSpan.FromSeconds(10));
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
Stage("dispose EGL");
|
||||||
|
rotator.Dispose();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Stage("decode");
|
||||||
|
using var retriever = new MediaMetadataRetriever();
|
||||||
|
retriever.SetDataSource(path);
|
||||||
|
using var bitmap = retriever.GetFrameAtTime(0, Option.ClosestSync)
|
||||||
|
?? throw new InvalidOperationException($"Cannot decode rotation {rotation}.");
|
||||||
|
if (bitmap.Width != outputWidth || bitmap.Height != outputHeight)
|
||||||
|
throw new InvalidOperationException($"Rotation {rotation}: dimensions {bitmap.Width}x{bitmap.Height}, expected {outputWidth}x{outputHeight}.");
|
||||||
|
Color[] expected = rotation switch
|
||||||
|
{
|
||||||
|
90 => [Color.Blue, Color.Red, Color.Yellow, Color.Green],
|
||||||
|
180 => [Color.Yellow, Color.Blue, Color.Green, Color.Red],
|
||||||
|
270 => [Color.Green, Color.Yellow, Color.Red, Color.Blue],
|
||||||
|
_ => [Color.Red, Color.Green, Color.Blue, Color.Yellow]
|
||||||
|
};
|
||||||
|
for (var i = 0; i < expected.Length; i++)
|
||||||
|
{
|
||||||
|
var actual = new Color(bitmap.GetPixel((i % 2 * 2 + 1) * outputWidth / 4,
|
||||||
|
(i / 2 * 2 + 1) * outputHeight / 4));
|
||||||
|
var target = expected[i];
|
||||||
|
if (Math.Abs(actual.R - target.R) > 40 || Math.Abs(actual.G - target.G) > 40 || Math.Abs(actual.B - target.B) > 40)
|
||||||
|
throw new InvalidOperationException($"Rotation {rotation}, quadrant {i}: {actual}, expected {target}.");
|
||||||
|
}
|
||||||
|
File.Delete(path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
#endif
|
||||||
|
|
@ -1,11 +1,13 @@
|
||||||
#if ANDROID
|
#if ANDROID
|
||||||
using System;
|
using System;
|
||||||
using System.IO;
|
using System.IO;
|
||||||
|
using System.Linq;
|
||||||
using System.Runtime.Versioning;
|
using System.Runtime.Versioning;
|
||||||
using System.Threading;
|
using System.Threading;
|
||||||
using System.Threading.Tasks;
|
using System.Threading.Tasks;
|
||||||
using Android.App;
|
using Android.App;
|
||||||
using Android.Content;
|
using Android.Content;
|
||||||
|
using Android.Media;
|
||||||
using Android.OS;
|
using Android.OS;
|
||||||
using Avalonia.Threading;
|
using Avalonia.Threading;
|
||||||
using PostIt.Android;
|
using PostIt.Android;
|
||||||
|
|
@ -34,7 +36,7 @@ public sealed class AndroidPodcastCaptureTestActivity : Activity
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await RunAsync(result);
|
await RunAsync(result);
|
||||||
File.WriteAllText(resultPath, $"PASS\nsegments={result.GetInt("segments")}\ndirectory={result.GetString("directory")}\nbackground={result.GetString("background")}");
|
File.WriteAllText(resultPath, $"PASS\nsegments={result.GetInt("segments")}\ndurationSeconds={result.GetDouble("durationSeconds")}\naudio={result.GetBoolean("audio")}\ndirectory={result.GetString("directory")}\nbackground={result.GetString("background")}\norientation={result.GetString("orientation")}\ntextureTransform={result.GetString("textureTransform")}");
|
||||||
}
|
}
|
||||||
catch (Exception ex)
|
catch (Exception ex)
|
||||||
{
|
{
|
||||||
|
|
@ -68,6 +70,11 @@ public sealed class AndroidPodcastCaptureTestActivity : Activity
|
||||||
await Task.Delay(9000);
|
await Task.Delay(9000);
|
||||||
var after = SegmentCount(directory);
|
var after = SegmentCount(directory);
|
||||||
if (capture.Completion.IsFaulted) await capture.Completion;
|
if (capture.Completion.IsFaulted) await capture.Completion;
|
||||||
|
if (capture is AndroidPodcastCapture androidCapture)
|
||||||
|
{
|
||||||
|
result.PutString("orientation", androidCapture.OrientationDiagnostics);
|
||||||
|
result.PutString("textureTransform", androidCapture.TextureTransform);
|
||||||
|
}
|
||||||
if (after <= before) throw new InvalidOperationException($"Capture did not continue in background: {before} -> {after} segments.");
|
if (after <= before) throw new InvalidOperationException($"Capture did not continue in background: {before} -> {after} segments.");
|
||||||
if (_notificationStop)
|
if (_notificationStop)
|
||||||
{
|
{
|
||||||
|
|
@ -81,7 +88,25 @@ public sealed class AndroidPodcastCaptureTestActivity : Activity
|
||||||
await capture.Completion.WaitAsync(TimeSpan.FromSeconds(15));
|
await capture.Completion.WaitAsync(TimeSpan.FromSeconds(15));
|
||||||
var segments = HlsCapturePlaylist.Parse(await File.ReadAllTextAsync(Path.Combine(directory, "index.m3u8")));
|
var segments = HlsCapturePlaylist.Parse(await File.ReadAllTextAsync(Path.Combine(directory, "index.m3u8")));
|
||||||
if (segments.Count < 3) throw new InvalidOperationException("Insufficient capture segments.");
|
if (segments.Count < 3) throw new InvalidOperationException("Insufficient capture segments.");
|
||||||
|
var duration = segments.Sum(segment => segment.DurationSeconds);
|
||||||
|
if (duration < 8 || duration > 16)
|
||||||
|
throw new InvalidOperationException($"Expected 8-16 seconds of capture, got {duration}.");
|
||||||
|
using var extractor = new MediaExtractor();
|
||||||
|
extractor.SetDataSource(Path.Combine(directory, segments[0].FileName));
|
||||||
|
var video = false;
|
||||||
|
var audio = false;
|
||||||
|
for (var track = 0; track < extractor.TrackCount; track++)
|
||||||
|
{
|
||||||
|
using var format = extractor.GetTrackFormat(track);
|
||||||
|
var mime = format.GetString(MediaFormat.KeyMime);
|
||||||
|
video |= mime == "video/avc";
|
||||||
|
audio |= mime == "audio/mp4a-latm";
|
||||||
|
}
|
||||||
|
if (!video || (_microphone && !audio))
|
||||||
|
throw new InvalidOperationException($"Missing capture tracks: H.264={video}, AAC={audio}.");
|
||||||
result.PutInt("segments", segments.Count);
|
result.PutInt("segments", segments.Count);
|
||||||
|
result.PutDouble("durationSeconds", duration);
|
||||||
|
result.PutBoolean("audio", audio);
|
||||||
result.PutString("directory", directory);
|
result.PutString("directory", directory);
|
||||||
result.PutString("background", $"segments {before} -> {after}");
|
result.PutString("background", $"segments {before} -> {after}");
|
||||||
}
|
}
|
||||||
|
|
|
||||||
37
src/PostIt/PostIt.Tests/CameraFrameClockTests.cs
Normal file
37
src/PostIt/PostIt.Tests/CameraFrameClockTests.cs
Normal file
|
|
@ -0,0 +1,37 @@
|
||||||
|
using PostIt.Android.Services;
|
||||||
|
|
||||||
|
namespace PostIt.Tests;
|
||||||
|
|
||||||
|
public sealed class CameraFrameClockTests
|
||||||
|
{
|
||||||
|
[Theory]
|
||||||
|
[InlineData(4_213_000_000_000L)]
|
||||||
|
[InlineData(0L)]
|
||||||
|
[InlineData(-1_000_000_000L)]
|
||||||
|
public void Aligns_camera_epoch_to_encoder_and_preserves_frame_intervals(long cameraOffset)
|
||||||
|
{
|
||||||
|
var clock = new CameraFrameClock();
|
||||||
|
const long now = 20_000_000_000_000L;
|
||||||
|
var camera = now + cameraOffset;
|
||||||
|
Assert.Equal(now, clock.GetPresentationTime(camera, now));
|
||||||
|
Assert.Equal(now + 33_333_333, clock.GetPresentationTime(camera + 33_333_333, now + 40_000_000));
|
||||||
|
Assert.Equal(now + 100_000_000, clock.GetPresentationTime(camera + 100_000_000, now + 110_000_000));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Does_not_submit_the_same_camera_image_twice()
|
||||||
|
{
|
||||||
|
var clock = new CameraFrameClock();
|
||||||
|
Assert.Equal(100, clock.GetPresentationTime(1000, 100));
|
||||||
|
Assert.Null(clock.GetPresentationTime(1000, 110));
|
||||||
|
Assert.Equal(120, clock.GetPresentationTime(1020, 130));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Reports_a_camera_clock_that_goes_backwards()
|
||||||
|
{
|
||||||
|
var clock = new CameraFrameClock();
|
||||||
|
clock.GetPresentationTime(1000, 100);
|
||||||
|
Assert.Throws<InvalidOperationException>(() => clock.GetPresentationTime(999, 110));
|
||||||
|
}
|
||||||
|
}
|
||||||
61
src/PostIt/PostIt.Tests/CameraFrameOrientationTests.cs
Normal file
61
src/PostIt/PostIt.Tests/CameraFrameOrientationTests.cs
Normal file
|
|
@ -0,0 +1,61 @@
|
||||||
|
using PostIt.Android.Services;
|
||||||
|
|
||||||
|
namespace PostIt.Tests;
|
||||||
|
|
||||||
|
public sealed class CameraFrameOrientationTests
|
||||||
|
{
|
||||||
|
[Theory]
|
||||||
|
[InlineData(90, 0, false, 90)]
|
||||||
|
[InlineData(90, 90, false, 0)]
|
||||||
|
[InlineData(90, 180, false, 270)]
|
||||||
|
[InlineData(90, 270, false, 180)]
|
||||||
|
[InlineData(270, 0, true, 270)]
|
||||||
|
[InlineData(270, 90, true, 0)]
|
||||||
|
[InlineData(270, 180, true, 90)]
|
||||||
|
[InlineData(270, 270, true, 180)]
|
||||||
|
[InlineData(0, 90, false, 270)]
|
||||||
|
[InlineData(180, 270, true, 90)]
|
||||||
|
public void Uses_sensor_display_and_lens_facing(int sensor, int display, bool front, int expected)
|
||||||
|
=> Assert.Equal(expected, CameraFrameOrientation.GetRotation(sensor, display, front));
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(0, new float[] { 0, 0, 1, 0, 0, 1, 1, 1 })]
|
||||||
|
[InlineData(90, new float[] { 1, 0, 1, 1, 0, 0, 0, 1 })]
|
||||||
|
[InlineData(180, new float[] { 1, 1, 0, 1, 1, 0, 0, 0 })]
|
||||||
|
[InlineData(270, new float[] { 0, 1, 0, 0, 1, 1, 1, 0 })]
|
||||||
|
public void Maps_output_corners_to_input_for_clockwise_rotation(int rotation, float[] expected)
|
||||||
|
=> Assert.Equal(expected, CameraFrameOrientation.GetTextureCoordinates(rotation));
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(90, 0, -1, 0)]
|
||||||
|
[InlineData(270, 0, 1, 0)]
|
||||||
|
[InlineData(180, -1, 0, 0)]
|
||||||
|
[InlineData(90, 1, 0, 90)]
|
||||||
|
[InlineData(270, 1, 0, 270)]
|
||||||
|
[InlineData(0, 0, -1, 270)]
|
||||||
|
[InlineData(180, 0, -1, 90)]
|
||||||
|
public void Does_not_repeat_rotation_already_in_surface_texture(
|
||||||
|
int desired, float x, float y, int expected)
|
||||||
|
{
|
||||||
|
float[] matrix = [x, y, 0, 0, y, -x, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1];
|
||||||
|
Assert.Equal(expected, CameraFrameOrientation.GetRemainingRotation(desired, matrix));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Recognizes_the_measured_rear_camera_transform_even_with_cropping()
|
||||||
|
{
|
||||||
|
float[] matrix = [0, -0.9f, 0, 0, -0.8f, 0, 0, 0, 0, 0, 1, 0, 0.9f, 0.95f, 0, 1];
|
||||||
|
Assert.Equal(0, CameraFrameOrientation.GetRemainingRotation(90, matrix));
|
||||||
|
}
|
||||||
|
|
||||||
|
[Theory]
|
||||||
|
[InlineData(-90)]
|
||||||
|
[InlineData(45)]
|
||||||
|
[InlineData(360)]
|
||||||
|
public void Rejects_invalid_angles(int rotation)
|
||||||
|
{
|
||||||
|
Assert.Throws<ArgumentOutOfRangeException>(() => CameraFrameOrientation.GetRotation(rotation, 0, false));
|
||||||
|
Assert.Throws<ArgumentOutOfRangeException>(() => CameraFrameOrientation.GetRotation(90, rotation, false));
|
||||||
|
Assert.Throws<ArgumentOutOfRangeException>(() => CameraFrameOrientation.GetTextureCoordinates(rotation));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
@ -25,5 +25,8 @@
|
||||||
<ItemGroup>
|
<ItemGroup>
|
||||||
<Using Include="Xunit" />
|
<Using Include="Xunit" />
|
||||||
</ItemGroup>
|
</ItemGroup>
|
||||||
<ItemGroup></ItemGroup>
|
<ItemGroup>
|
||||||
|
<Compile Include="..\PostIt.Android\Services\CameraFrameOrientation.cs" Link="Android\CameraFrameOrientation.cs" />
|
||||||
|
<Compile Include="..\PostIt.Android\Services\CameraFrameClock.cs" Link="Android\CameraFrameClock.cs" />
|
||||||
|
</ItemGroup>
|
||||||
</Project>
|
</Project>
|
||||||
Loading…
Reference in a new issue