feat(postit): add live capture previews on Android and Desktop

Reuse oriented Android EGL frames and duplicate Desktop video output before encoding without microphone monitoring. Cover preview lifecycle, rendering and recording with targeted tests and document the feature.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Paul Schneider 2026-10-25 01:03:36 +00:00
commit c2068f1476
Signed by: notazof
GPG key ID: AD4B9B545FBD7D12
14 changed files with 469 additions and 27 deletions

View file

@ -5,6 +5,7 @@
### Added
* [PostIt] Show a local live capture preview on Android and Desktop without opening a second camera or reading back uploaded segments. Share Android's oriented EGL frames with an aspect-preserving preview surface; duplicate Desktop LibVLC output without local microphone playback.
* [Podcasts] Prepare authenticated, seekable MP4 replays for new and existing Completed broadcasts using server-side FFmpeg stream copy and faststart. Retain MPEG-TS, account for both copies in the owner quota, recover preparation after restarts, and expose owner-only retries plus GET/HEAD byte-range playback.
* [PostIt] Add Android/Desktop native video overlays for MP4 replay position, duration, seeking and play/pause, with three-second auto-hide. Use bounded authenticated range input with token refresh; preserve MPEG-TS live/interrupted playback.

View file

@ -83,6 +83,13 @@ Install LibVLC 3 with the `livehttp`, MPEG-TS, x264 and AAC plugins on the
Desktop machine. Windows capture uses DirectShow; Linux uses V4L2 and the
default PulseAudio microphone. FFmpeg is **not** a client dependency.
Select a camera and optionally a microphone, enter a title, then start.
An **Apercu local de la capture** appears at the top of the Podcasts page on
Android and Desktop. It uses the current capture, not a second camera session
or the server replay. Android displays the same oriented frames as the encoder,
with black bars to preserve proportions; Desktop duplicates LibVLC's video
output. Local microphone playback is disabled to avoid acoustic feedback.
Stopping releases the preview; hiding/recreating the Android preview surface
does not restart the encoder or change its timestamps.
Stopping publishes the retained replay; navigating away or logging out
stops capture too. A second signed-in PostIt instance can watch the live
broadcast while it is being captured.

View file

@ -84,6 +84,8 @@ dotnet test --project src/test/Yavsc.Api.Tests/Yavsc.Api.Tests.csproj \
dotnet test --project src/PostIt/PostIt.Tests/PostIt.Tests.csproj \
--filter-class '*PodcastTests' --filter-class '*PodcastReplayTests' \
--filter-class '*YavscApiClientTests' --filter-class '*PodcastNativeCaptureTests'
dotnet test --project src/PostIt/PostIt.Tests/PostIt.Tests.csproj \
--filter-method '*Shared_page_exposes_running_capture_preview_and_clears_it_before_disposal'
```
Les tests serveur couvrent HTTP/authentification, proprietaire, sequences,
@ -108,9 +110,15 @@ Pour les activer, installer LibVLC avec ses encodeurs H.264/AAC et definir
cette variable sur un fichier video local d'environ huit secondes contenant
une piste audio. Ils exercent la segmentation LibVLC reelle, l'envoi et la
reprise apres acquittement perdu, le dernier segment apres arret et le
decodage authentifie du replay MPEG-TS. Aucun FFmpeg n'est requis par ces tests
natifs client ni par PostIt (le serveur MP4, lui, en a besoin).
decodage authentifie du replay MPEG-TS. FFmpeg n'est pas une dependance
d'execution de PostIt (le serveur MP4, lui, en a besoin).
Les tests materiels camera/microphone Windows/Linux restent manuels.
Les deux variantes du test d'encodage HLS (avec/sans apercu) necessitent
egalement `ffprobe` pour verifier independamment les pistes H.264 et AAC
du premier segment. La variante avec apercu attend une image decodee
par LibVLC dans un tampon memoire, sans ouvrir de camera ni diffuser le son
sur les haut-parleurs. Le test de page headless verifie aussi le binding
du controle d'apercu et son retrait avant liberation de la capture.
### Rotation de la capture Android
@ -137,6 +145,9 @@ Attendre la fin du test : `RUNNING` doit devenir `PASS`. Il encode quatre
quadrants colores en MPEG-TS pour 0, 90, 180 et 270 degres, decode une image et
verifie les dimensions (640x480 ou 480x640) et les couleurs aux quatre positions.
La tolerance de chaque composante RGB est de 40 pour tenir compte de H.264.
Il verifie aussi les couleurs RGBA de l'apercu OpenGL (tolerance 5), les
bandes noires sans deformation et le detachement/rattachement de sa surface
pendant l'encodage, toujours pour les quatre rotations.
Une erreur de rendu, d'encodage, de decodage ou de comparaison produit `FAIL`
avec l'exception. Cette activite est exclue des builds Release.

View file

@ -37,9 +37,17 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
private readonly Action<Exception> _failed;
private readonly float[] _textureMatrix = new float[16];
private readonly CameraFrameClock _clock = new();
private readonly object _lifecycle = new();
private EGLDisplay? _display;
private EGLConfig? _config;
private EGLContext? _context;
private EGLSurface? _window;
private EGLSurface? _previewWindow;
private Action<Exception>? _previewFailed;
private int _previewWidth;
private int _previewHeight;
private int _outputWidth;
private int _outputHeight;
private SurfaceTexture? _texture;
private FrameListener? _listener;
private Surface? _cameraSurface;
@ -98,6 +106,7 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
var count = new int[1];
if (!EGL14.EglChooseConfig(_display, attributes, 0, configs, 0, 1, count, 0) || count[0] == 0)
throw EglError("Configuration EGL pour l'encodeur");
_config = configs[0];
_context = EGL14.EglCreateContext(_display, configs[0], EGL14.EglNoContext,
[EGL14.EglContextClientVersion, 2, EGL14.EglNone], 0);
if (_context is null || _context.Equals(EGL14.EglNoContext))
@ -124,7 +133,9 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
BindAttribute("textureCoordinate", _coordinates);
_matrixLocation = GLES20.GlGetUniformLocation(_program, "textureMatrix");
GLES20.GlUniform1i(GLES20.GlGetUniformLocation(_program, "cameraTexture"), 0);
GLES20.GlViewport(0, 0, rotation is 90 or 270 ? height : width, rotation is 90 or 270 ? width : height);
_outputWidth = rotation is 90 or 270 ? height : width;
_outputHeight = rotation is 90 or 270 ? width : height;
GLES20.GlViewport(0, 0, _outputWidth, _outputHeight);
CheckGl();
_texture = new SurfaceTexture(_textureId);
@ -136,11 +147,33 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
internal Task StartAsync() => InvokeAsync(() => _rendering = true);
internal Task SetPreviewAsync(Surface? surface, int width = 0, int height = 0, Action<Exception>? failed = null)
{
lock (_lifecycle)
{
if (_disposed) return Task.CompletedTask;
return InvokeAsync(() =>
{
ReleasePreview();
if (surface is null) return;
_previewWindow = EGL14.EglCreateWindowSurface(_display, _config, surface, [EGL14.EglNone], 0);
if (_previewWindow is null || _previewWindow.Equals(EGL14.EglNoSurface))
throw EglError("Creation de la surface d'apercu");
_previewWidth = width;
_previewHeight = height;
_previewFailed = failed;
});
}
}
private void DrawFrame(SurfaceTexture texture)
{
if (!_rendering) return;
try
{
if (!EGL14.EglMakeCurrent(_display, _window, _window, _context))
throw EglError("Activation de l'encodeur");
GLES20.GlViewport(0, 0, _outputWidth, _outputHeight);
texture.UpdateTexImage();
var presentationTime = _clock.GetPresentationTime(texture.Timestamp, Java.Lang.JavaSystem.NanoTime());
if (presentationTime is null) return;
@ -163,6 +196,7 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
throw EglError("Horodatage de l'image camera");
if (!EGL14.EglSwapBuffers(_display, _window))
throw EglError("Envoi de l'image a l'encodeur");
DrawPreview();
}
catch (Exception ex)
{
@ -171,6 +205,47 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
}
}
private void DrawPreview()
{
if (_previewWindow is null || _previewWidth <= 0 || _previewHeight <= 0) return;
try
{
if (!EGL14.EglMakeCurrent(_display, _previewWindow, _previewWindow, _context))
throw EglError("Activation de l'apercu");
GLES20.GlClearColor(0, 0, 0, 1);
GLES20.GlClear(GLES20.GlColorBufferBit);
var scale = Math.Min((double)_previewWidth / _outputWidth, (double)_previewHeight / _outputHeight);
var width = Math.Max(1, (int)(_outputWidth * scale));
var height = Math.Max(1, (int)(_outputHeight * scale));
GLES20.GlViewport((_previewWidth - width) / 2, (_previewHeight - height) / 2, width, height);
GLES20.GlDrawArrays(GLES20.GlTriangleStrip, 0, 4);
CheckGl();
if (!EGL14.EglSwapBuffers(_display, _previewWindow))
throw EglError("Affichage de l'apercu");
}
catch (Exception ex)
{
var failed = _previewFailed;
global::Android.Util.Log.Error("PostIt.Podcast", $"Apercu indisponible : {ex}");
ReleasePreview();
failed?.Invoke(ex);
}
}
private void ReleasePreview()
{
if (_previewWindow is not null)
{
// Never leave a surface current when its Android holder is about to destroy it.
EGL14.EglMakeCurrent(_display, _window, _window, _context);
if (!_previewWindow.Equals(EGL14.EglNoSurface))
EGL14.EglDestroySurface(_display, _previewWindow);
_previewWindow.Dispose();
_previewWindow = null;
}
_previewFailed = null;
}
private Task InvokeAsync(Action action)
{
var completion = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
@ -257,9 +332,14 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
public void Dispose()
{
if (_disposed) return;
_disposed = true;
try { InvokeAsync(Release).GetAwaiter().GetResult(); }
Task release;
lock (_lifecycle)
{
if (_disposed) return;
_disposed = true;
release = InvokeAsync(Release);
}
try { release.GetAwaiter().GetResult(); }
finally
{
_thread.QuitSafely();
@ -272,6 +352,7 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
private void Release()
{
_rendering = false;
ReleasePreview();
_texture?.SetOnFrameAvailableListener(null);
_cameraSurface?.Release();
_cameraSurface?.Dispose();
@ -293,6 +374,7 @@ internal sealed class AndroidCameraFrameRotator : IDisposable
EGL14.EglTerminate(_display);
}
_window?.Dispose();
_config?.Dispose();
_context?.Dispose();
_display?.Dispose();
}

View file

@ -0,0 +1,90 @@
using System;
using Android.Views;
using Avalonia.Android;
using Avalonia.Controls;
using Avalonia.Platform;
using Avalonia.Threading;
namespace PostIt.Android.Services;
internal sealed class AndroidCapturePreview : Grid, IDisposable
{
private readonly PreviewHost _host;
private readonly TextBlock _status = new() { TextWrapping = Avalonia.Media.TextWrapping.Wrap };
internal AndroidCapturePreview(AndroidCameraFrameRotator rotator)
{
RowDefinitions = new RowDefinitions("*,Auto");
_host = new PreviewHost(rotator, ReportFailure);
Children.Add(_host);
SetRow(_status, 1);
Children.Add(_status);
}
private void ReportFailure(Exception error)
{
global::Android.Util.Log.Error("PostIt.Podcast", $"Apercu indisponible : {error}");
Dispatcher.UIThread.Post(() => _status.Text = $"Apercu indisponible : {error.Message}. La capture continue.");
}
public void Dispose() => _host.Dispose();
private sealed class PreviewHost(AndroidCameraFrameRotator rotator, Action<Exception> failed)
: NativeControlHost, IDisposable
{
private SurfaceView? _view;
private SurfaceCallback? _callback;
private bool _disposed;
protected override IPlatformHandle CreateNativeControlCore(IPlatformHandle parent)
{
var activity = MainActivity.Current
?? throw new InvalidOperationException("Activite Android indisponible pour l'apercu.");
_view = new SurfaceView(activity);
_callback = new SurfaceCallback(this);
_view.Holder!.AddCallback(_callback);
return new AndroidViewControlHandle(_view);
}
private void SetSurface(ISurfaceHolder holder, int width, int height)
{
if (_disposed) return;
try { rotator.SetPreviewAsync(holder.Surface, width, height, failed).GetAwaiter().GetResult(); }
catch (Exception ex) { failed(ex); }
}
private void Detach()
{
try { rotator.SetPreviewAsync(null).GetAwaiter().GetResult(); }
catch (Exception ex) { failed(ex); }
}
protected override void DestroyNativeControlCore(IPlatformHandle control)
{
Detach();
if (_callback is not null)
{
_view?.Holder?.RemoveCallback(_callback);
_callback.Dispose();
_callback = null;
}
_view = null;
base.DestroyNativeControlCore(control);
}
public void Dispose()
{
if (_disposed) return;
_disposed = true;
Detach();
}
private sealed class SurfaceCallback(PreviewHost owner) : Java.Lang.Object, ISurfaceHolderCallback
{
public void SurfaceCreated(ISurfaceHolder holder) { }
public void SurfaceChanged(ISurfaceHolder holder, global::Android.Graphics.Format format, int width, int height)
=> owner.SetSurface(holder, width, height);
public void SurfaceDestroyed(ISurfaceHolder holder) => owner.Detach();
}
}
}

View file

@ -27,6 +27,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
private CameraCaptureSession? _session;
private MediaRecorder? _recorder;
private AndroidCameraFrameRotator? _rotator;
private AndroidCapturePreview? _preview;
public Avalonia.Controls.Control? Preview => _preview;
private Surface? _surface;
private ParcelFileDescriptor? _readPipe;
private ParcelFileDescriptor? _writePipe;
@ -160,6 +162,7 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
_surface = _recorder.Surface ?? throw new InvalidOperationException("Surface encodeur indisponible.");
_rotator = await AndroidCameraFrameRotator.CreateAsync(_surface, size.Width, size.Height, rotation, Fail);
_preview = new AndroidCapturePreview(_rotator);
ct.ThrowIfCancellationRequested();
if (_stopRequested) throw new System.OperationCanceledException("Demarrage de la capture annule.");
var cameraSurface = _rotator.CameraSurface;
@ -286,6 +289,8 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
{
if (_stopped) return;
_stopped = true;
if (_preview is not null)
await Avalonia.Threading.Dispatcher.UIThread.InvokeAsync(_preview.Dispose);
await Task.Run(StopNative);
await _pump;
}

View file

@ -27,7 +27,7 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
{
foreach (var rotation in new[] { 0, 90, 180, 270 })
await VerifyRotationAsync(rotation);
File.WriteAllText(resultPath, "PASS\nrotations=0,90,180,270\nformat=MPEG-TS\nsource=synthetic");
File.WriteAllText(resultPath, "PASS\nrotations=0,90,180,270\nformat=MPEG-TS\npreview=RGBA,detach,reattach,letterbox\nsource=synthetic");
}
catch (Exception ex)
{
@ -60,6 +60,10 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
recorder.Prepare();
using var encoderSurface = recorder.Surface
?? throw new InvalidOperationException("Encoder surface unavailable.");
using var preview = ImageReader.NewInstance(outputWidth + 160, outputHeight + 80,
(ImageFormatType)Format.Rgba8888, 3)
?? throw new InvalidOperationException("Preview reader unavailable.");
var previewFrames = 0;
Exception? failure = null;
var stopping = 0;
Stage("initialize EGL");
@ -71,6 +75,8 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
File.WriteAllText(Path.Combine(FilesDir.AbsolutePath, "camera-rotation-test-result.txt"), $"FAIL\n{error}");
}))
{
await rotator.SetPreviewAsync(preview.Surface, preview.Width, preview.Height,
error => failure = error);
Stage("start recorder");
recorder.Start();
Task? stop = null;
@ -83,6 +89,10 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
for (var frame = 0; frame < 90; frame++)
{
if (failure is not null) throw failure;
if (frame == 10) await rotator.SetPreviewAsync(null);
if (frame == 15)
await rotator.SetPreviewAsync(preview.Surface, preview.Width, preview.Height,
error => failure = error);
if (frame == 45)
{
Volatile.Write(ref stopping, 1);
@ -107,11 +117,24 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
canvas.Dispose();
}
await Task.Delay(34);
using var image = preview.AcquireLatestImage();
if (image is not null)
{
try
{
VerifyPreview(image, rotation, outputWidth, outputHeight);
previewFrames++;
}
finally { image.Close(); }
}
}
if (failure is not null) throw failure;
Stage("wait for recorder stop");
await (stop ?? throw new InvalidOperationException("Recorder stop was not requested."))
.WaitAsync(TimeSpan.FromSeconds(10));
if (previewFrames < 8)
throw new InvalidOperationException($"Too few preview frames: {previewFrames}.");
await rotator.SetPreviewAsync(null);
}
finally
{
@ -143,5 +166,33 @@ public sealed class AndroidCameraFrameRotationTestActivity : Activity
}
File.Delete(path);
}
private static void VerifyPreview(Image image, int rotation, int outputWidth, int outputHeight)
{
var plane = image.GetPlanes()![0];
var buffer = plane.Buffer!;
var scale = Math.Min((double)image.Width / outputWidth, (double)image.Height / outputHeight);
var width = (int)(outputWidth * scale);
var height = (int)(outputHeight * scale);
Color[] expected = rotation switch
{
90 => [Color.Blue, Color.Red, Color.Yellow, Color.Green],
180 => [Color.Yellow, Color.Blue, Color.Green, Color.Red],
270 => [Color.Green, Color.Yellow, Color.Red, Color.Blue],
_ => [Color.Red, Color.Green, Color.Blue, Color.Yellow]
};
for (var i = 0; i < expected.Length; i++)
{
var x = (image.Width - width) / 2 + (i % 2 * 2 + 1) * width / 4;
var y = (image.Height - height) / 2 + (i / 2 * 2 + 1) * height / 4;
var offset = y * plane.RowStride + x * plane.PixelStride;
var actual = new Color(buffer.Get(offset) & 255, buffer.Get(offset + 1) & 255, buffer.Get(offset + 2) & 255);
var target = expected[i];
if (Math.Abs(actual.R - target.R) > 5 || Math.Abs(actual.G - target.G) > 5 || Math.Abs(actual.B - target.B) > 5)
throw new InvalidOperationException($"Preview rotation {rotation}, quadrant {i}: {actual}, expected {target}.");
}
if (buffer.Get(0) != 0 || buffer.Get(1) != 0 || buffer.Get(2) != 0)
throw new InvalidOperationException("Preview letterbox is not black.");
}
}
#endif

View file

@ -1,7 +1,10 @@
using System.Net;
using Avalonia.Headless.XUnit;
using Avalonia.Headless;
using Avalonia.Controls;
using PostIt.Services;
using PostIt.ViewModels;
using PostIt.Views;
using Yavsc.Abstract.Streaming;
using Yavsc.Api.Client;
@ -9,6 +12,69 @@ namespace PostIt.Tests;
public sealed class PodcastBroadcasterTests
{
[Fact]
public Task Shared_page_exposes_running_capture_preview_and_clears_it_before_disposal()
=> HeadlessUnitTestSession.GetOrStartForAssembly(typeof(PodcastBroadcasterTests).Assembly).Dispatch(async () =>
{
var tokenPath = Path.Combine(Path.GetTempPath(), $"postit-preview-{Guid.NewGuid():N}.json");
var devices = Platform.GetPodcastCaptureDevices;
var permissions = Platform.RequestPodcastPermissionsAsync;
var factory = Platform.CreatePodcastCaptureAsync;
var store = new TokenStore(tokenPath);
store.Save(new RefreshTokenRecord("test-access-token", "test-refresh-token", DateTimeOffset.UtcNow.AddHours(1), null));
await using var api = new YavscApiClient(new Settings(), store);
await using var transport = new Transport();
FakeCapture? capture = null;
PodcastsPageViewModel? vm = null;
try
{
var preview = new Border();
var disposed = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
Platform.GetPodcastCaptureDevices = () => new([new("Camera", "camera")], [new("None", "")]);
Platform.RequestPodcastPermissionsAsync = (_, _) => Task.CompletedTask;
Platform.CreatePodcastCaptureAsync = (_, _, directory, _) =>
{
capture = new FakeCapture(directory, preview)
{
OnDispose = () =>
{
Assert.Null(vm!.CapturePreview);
disposed.TrySetResult();
}
};
return Task.FromResult<IPodcastCapture>(capture);
};
var session = new SessionStatusViewModel { Api = api };
session.Refresh();
vm = new PodcastsPageViewModel(new PodcastApiClient(transport, () => "https://api.example/"), api, session)
{
Title = "Preview"
};
vm.DetectDevicesCommand.Execute(null);
await vm.StartCommand.ExecuteAsync(null);
Assert.Same(preview, vm.CapturePreview);
Assert.True(vm.IsBroadcasting);
var page = new PodcastsPage { DataContext = vm };
var host = page.FindControl<ContentControl>("CapturePreviewHost");
Assert.NotNull(host);
Assert.Same(preview, host.Content);
await vm.DeactivateAsync();
await disposed.Task.WaitAsync(TimeSpan.FromSeconds(5), TestContext.Current.CancellationToken);
Assert.Null(vm.CapturePreview);
Assert.Null(host.Content);
Assert.True(capture!.Stopped);
return true;
}
finally
{
if (vm?.IsBroadcasting == true) await vm.StopAsync();
Platform.GetPodcastCaptureDevices = devices;
Platform.RequestPodcastPermissionsAsync = permissions;
Platform.CreatePodcastCaptureAsync = factory;
File.Delete(tokenPath);
}
}, TestContext.Current.CancellationToken);
[AvaloniaFact]
public async Task Shared_page_uses_platform_devices_and_requests_permissions_before_creating_server_broadcast()
{
@ -120,9 +186,12 @@ public sealed class PodcastBroadcasterTests
public string Directory { get; }
public bool Stopped { get; private set; }
public bool Disposed { get; private set; }
public Control? Preview { get; }
public Action? OnDispose { get; init; }
public Task Completion => _completion.Task;
public FakeCapture(string directory)
public FakeCapture(string directory, Control? preview = null)
{
Preview = preview;
Directory = directory;
File.WriteAllBytes(Path.Combine(directory, "segment-00000000.ts"), [0x47, 1]);
File.WriteAllText(Path.Combine(directory, "index.m3u8"), "#EXTM3U\n#EXTINF:2,\nsegment-00000000.ts\n");
@ -136,7 +205,7 @@ public sealed class PodcastBroadcasterTests
_completion.TrySetResult();
return Task.CompletedTask;
}
public ValueTask DisposeAsync() { Disposed = true; return ValueTask.CompletedTask; }
public ValueTask DisposeAsync() { Disposed = true; OnDispose?.Invoke(); return ValueTask.CompletedTask; }
}
private sealed class Transport : IYavscApiClient

View file

@ -1,5 +1,8 @@
using System.Net;
using System.Net.Http;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Text.Json;
using LibVLCSharp.Shared;
using PostIt.Services;
using Yavsc.Abstract.Streaming;
@ -7,28 +10,47 @@ using Yavsc.Api.Client;
namespace PostIt.Tests;
public sealed class PodcastNativeCaptureTests
public sealed class PodcastNativeCaptureTests(ITestOutputHelper output)
{
public static bool HasFixture => File.Exists(Environment.GetEnvironmentVariable("POSTIT_TEST_MEDIA"));
[Fact(Skip = "Set POSTIT_TEST_MEDIA to a local video with audio and install LibVLC with H.264/AAC encoders.", SkipUnless = nameof(HasFixture))]
public async Task LibVLC_produces_complete_H264_AAC_HLS_segments_without_external_encoder()
[Theory(Skip = "Set POSTIT_TEST_MEDIA to a local video with audio and install LibVLC with H.264/AAC encoders.", SkipUnless = nameof(HasFixture))]
[InlineData(false)]
[InlineData(true)]
public async Task LibVLC_produces_complete_H264_AAC_HLS_segments_without_external_encoder(bool preview)
{
var root = Path.Combine(Path.GetTempPath(), $"postit native's hls {Guid.NewGuid():N}");
Directory.CreateDirectory(root);
var pixels = preview ? Marshal.AllocHGlobal(640 * 480 * 4) : IntPtr.Zero;
try
{
Core.Initialize();
using var vlc = new LibVLC();
using var vlc = new LibVLC("--vout=dummy", "--aout=dummy", "--no-xlib",
"--avcodec-hw=none", "--avcodec-threads=2");
using var media = new Media(vlc, new Uri(Path.GetFullPath(Environment.GetEnvironmentVariable("POSTIT_TEST_MEDIA")!)));
using var player = new MediaPlayer(vlc);
PodcastBroadcaster.ConfigureOutput(media, root);
var displayed = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
if (preview)
{
player.SetVideoFormat("RV32", 640, 480, 640 * 4);
player.SetVideoCallbacks((_, planes) =>
{
Marshal.WriteIntPtr(planes, pixels);
return IntPtr.Zero;
}, null, (_, _) => displayed.TrySetResult());
}
PodcastBroadcaster.ConfigureOutput(media, root, preview);
var ended = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
player.EndReached += (_, _) => ended.TrySetResult();
player.EncounteredError += (_, _) => ended.TrySetException(new InvalidOperationException("LibVLC encoding failed."));
Assert.True(player.Play(media));
output.WriteLine($"Encoding started (preview={preview}).");
if (preview)
await displayed.Task.WaitAsync(TimeSpan.FromSeconds(10), TestContext.Current.CancellationToken);
await ended.Task.WaitAsync(TimeSpan.FromSeconds(45), TestContext.Current.CancellationToken);
output.WriteLine("End reached; stopping player.");
player.Stop();
output.WriteLine("Player stopped; checking segments.");
var playlist = await File.ReadAllTextAsync(Path.Combine(root, "index.m3u8"), TestContext.Current.CancellationToken);
var segments = HlsCapturePlaylist.Parse(playlist);
Assert.True(segments.Count >= 2, playlist);
@ -41,8 +63,34 @@ public sealed class PodcastNativeCaptureTests
Assert.InRange(segment.DurationSeconds, 0.001, 15);
for (var i = 0; i < bytes.Length; i += 188) Assert.Equal(0x47, bytes[i]);
}
var start = new ProcessStartInfo("ffprobe") { RedirectStandardOutput = true, RedirectStandardError = true };
foreach (var argument in new[] { "-v", "error", "-show_streams", "-of", "json", Path.Combine(root, segments[0].FileName) })
start.ArgumentList.Add(argument);
using var probe = Process.Start(start) ?? throw new IOException("Cannot start ffprobe.");
using var timeout = CancellationTokenSource.CreateLinkedTokenSource(TestContext.Current.CancellationToken);
timeout.CancelAfter(TimeSpan.FromSeconds(10));
var stdout = probe.StandardOutput.ReadToEndAsync(timeout.Token);
var stderr = probe.StandardError.ReadToEndAsync(timeout.Token);
try
{
await probe.WaitForExitAsync(timeout.Token);
Assert.True(probe.ExitCode == 0, await stderr);
using var json = JsonDocument.Parse(await stdout);
var streams = json.RootElement.GetProperty("streams").EnumerateArray().ToArray();
Assert.Contains(streams, track => track.GetProperty("codec_name").GetString() == "h264");
Assert.Contains(streams, track => track.GetProperty("codec_name").GetString() == "aac");
}
finally
{
if (!probe.HasExited) probe.Kill(entireProcessTree: true);
await probe.WaitForExitAsync(CancellationToken.None);
}
}
finally
{
if (pixels != IntPtr.Zero) Marshal.FreeHGlobal(pixels);
Directory.Delete(root, recursive: true);
}
finally { Directory.Delete(root, recursive: true); }
}
[Fact(Skip = "Set POSTIT_TEST_MEDIA to a local video with audio and install LibVLC with H.264/AAC encoders.", SkipUnless = nameof(HasFixture))]

View file

@ -1,6 +1,7 @@
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Avalonia.Controls;
namespace PostIt.Services;
@ -12,5 +13,6 @@ public sealed record PodcastCaptureDevices(
public interface IPodcastCapture : IAsyncDisposable
{
Task Completion { get; }
Control? Preview => null;
Task StopAsync();
}

View file

@ -1,6 +1,8 @@
using System;
using System.Threading.Tasks;
using LibVLCSharp.Shared;
using Avalonia.Controls;
using Avalonia.Threading;
namespace PostIt.Services;
@ -11,24 +13,37 @@ internal sealed class LibVlcPodcastCapture : IPodcastCapture
private readonly MediaPlayer _player;
private readonly TaskCompletionSource _completion = new(TaskCreationOptions.RunContinuationsAsynchronously);
private bool _stopped;
private bool _startScheduled;
private readonly LibVLCSharp.Avalonia.VideoView? _preview;
public Task Completion => _completion.Task;
public Control? Preview => _preview;
public LibVlcPodcastCapture(Func<LibVLC, Media> createMedia, string directory)
public LibVlcPodcastCapture(Func<LibVLC, Media> createMedia, string directory, bool preview = false)
{
Core.Initialize();
_vlc = CameraStreamer.CreatePreferredLibVlc();
// The usual Linux capture instance has a dummy video output; preview needs a drawable.
_vlc = preview ? new LibVLC("--avcodec-hw=none", "--aout=dummy") : CameraStreamer.CreatePreferredLibVlc();
try
{
_media = createMedia(_vlc);
PodcastBroadcaster.ConfigureOutput(_media, directory);
PodcastBroadcaster.ConfigureOutput(_media, directory, preview);
_player = new MediaPlayer(_vlc);
if (preview)
{
_preview = new LibVLCSharp.Avalonia.VideoView { MediaPlayer = _player };
_preview.LayoutUpdated += OnPreviewLayout;
}
_player.EncounteredError += OnError;
if (!_player.Play(_media))
throw new InvalidOperationException("LibVLC ne peut pas demarrer la capture. Verifiez la camera et les encodeurs H.264/AAC.");
if (!preview) StartPlayer();
}
catch
{
if (_preview is not null)
{
_preview.LayoutUpdated -= OnPreviewLayout;
_preview.MediaPlayer = null;
}
_player?.Dispose();
_media?.Dispose();
_vlc.Dispose();
@ -36,14 +51,53 @@ internal sealed class LibVlcPodcastCapture : IPodcastCapture
}
}
private void StartPlayer()
{
if (!_player.Play(_media))
throw new InvalidOperationException("LibVLC ne peut pas demarrer la capture. Verifiez la camera et les encodeurs H.264/AAC.");
}
private void OnPreviewLayout(object? sender, EventArgs e)
{
if (_startScheduled || _stopped || _preview is not { IsEffectivelyVisible: true } view
|| view.Bounds.Width <= 0 || view.Bounds.Height <= 0) return;
_startScheduled = true;
view.LayoutUpdated -= OnPreviewLayout;
// LibVLC must receive its native drawable before the display branch starts.
Dispatcher.UIThread.Post(() =>
{
if (_stopped) return;
try
{
view.TryUpdateNativeControlPosition();
StartPlayer();
}
catch (Exception ex) { _completion.TrySetException(ex); }
}, DispatcherPriority.Loaded);
}
private void OnError(object? sender, EventArgs e)
=> _completion.TrySetException(new InvalidOperationException(
"La capture LibVLC a echoue. Verifiez camera, microphone et encodeurs H.264/AAC."));
public async Task StopAsync()
{
if (_stopped) return;
_stopped = true;
if (_preview is not null)
{
var shouldStop = await Dispatcher.UIThread.InvokeAsync(() =>
{
if (_stopped) return false;
_stopped = true;
_preview.LayoutUpdated -= OnPreviewLayout;
return true;
});
if (!shouldStop) return;
}
else
{
if (_stopped) return;
_stopped = true;
}
await Task.Run(_player.Stop);
_completion.TrySetResult();
}
@ -53,6 +107,8 @@ internal sealed class LibVlcPodcastCapture : IPodcastCapture
try { await StopAsync(); }
finally
{
if (_preview is not null)
await Dispatcher.UIThread.InvokeAsync(() => _preview.MediaPlayer = null);
_player.EncounteredError -= OnError;
_player.Dispose();
_media.Dispose();

View file

@ -8,6 +8,7 @@ using System.Threading;
using System.Threading.Tasks;
using System.Text.Json;
using LibVLCSharp.Shared;
using Avalonia.Controls;
using Yavsc.Abstract.Streaming;
using Yavsc.Api.Client;
@ -24,6 +25,7 @@ public sealed class PodcastBroadcaster : IAsyncDisposable
private bool _disposed;
public PodcastResponse Podcast { get; private set; }
public Control? Preview => _capture.Preview;
public Task Completion { get; private set; } = Task.CompletedTask;
public event Action<PodcastResponse>? Progress;
@ -36,18 +38,24 @@ public sealed class PodcastBroadcaster : IAsyncDisposable
_directory = directory;
}
public static void ConfigureOutput(Media media, string directory)
public static void ConfigureOutput(Media media, string directory, bool preview = false)
{
var path = directory.Replace('\\', '/').Replace("\"", "\\\"");
media.AddOption(":sout=#transcode{vcodec=h264,venc=x264{preset=ultrafast,tune=zerolatency,keyint=60},vb=1200,fps=30,"
+ "acodec=mp4a,ab=128,channels=2,samplerate=48000}:std{"
var recording = "std{"
+ $"access=livehttp{{seglen=2,delsegs=false,numsegs=0,initial-segment-number=0,index=\"{path}/index.m3u8\","
+ $"index-url=segment-########.ts}},mux=ts{{use-key-frames}},dst=\"{path}/segment-########.ts\"}}");
+ $"index-url=segment-########.ts}},mux=ts{{use-key-frames}},dst=\"{path}/segment-########.ts\"}}";
var encoding = "transcode{vcodec=h264,venc=x264{preset=ultrafast,tune=zerolatency,keyint=60},vb=1200,fps=30,"
+ "acodec=mp4a,ab=128,channels=2,samplerate=48000}:"
+ recording;
var destination = encoding.Replace("\\", "\\\\").Replace("\"", "\\\"");
media.AddOption(":sout=#" + (preview ? $"duplicate{{dst=display{{noaudio}},dst=\"{destination}\"}}" : encoding));
}
public static async Task<PodcastBroadcaster> StartAsync(PodcastApiClient api, string title,
string camera, string? microphone, CancellationToken ct)
=> await StartAsync(api, title, vlc => new CameraStreamer(vlc).CreateCaptureMedia(camera, microphone), ct);
=> await StartAsync(api, title,
(directory, _) => Task.FromResult<IPodcastCapture>(new LibVlcPodcastCapture(
vlc => new CameraStreamer(vlc).CreateCaptureMedia(camera, microphone), directory, preview: true)), ct);
internal static async Task<PodcastBroadcaster> StartAsync(PodcastApiClient api, string title,
Func<LibVLC, Media> createMedia, CancellationToken ct)

View file

@ -4,6 +4,7 @@ using System.Net.Http;
using System.Threading;
using System.Threading.Tasks;
using Avalonia;
using Avalonia.Controls;
using Avalonia.Threading;
using CommunityToolkit.Mvvm.ComponentModel;
using CommunityToolkit.Mvvm.Input;
@ -30,6 +31,8 @@ public partial class PodcastsPageViewModel : ViewModelBase
private CaptureDevice? _camera;
private CaptureDevice? _microphone;
private PodcastResponse? _selected;
private Control? _capturePreview;
public Control? CapturePreview { get => _capturePreview; private set => SetProperty(ref _capturePreview, value); }
public ObservableCollection<PodcastResponse> Podcasts { get; } = new();
public ObservableCollection<CaptureDevice> Cameras { get; } = new();
@ -194,6 +197,7 @@ public partial class PodcastsPageViewModel : ViewModelBase
(directory, ct) => createCapture(camera, microphone, directory, ct), _pageCancellation.Token)
: await PodcastBroadcaster.StartAsync(_client, Title, camera, microphone, _pageCancellation.Token);
_broadcaster.Progress += OnProgress;
CapturePreview = _broadcaster.Preview;
_session.LogoutCompleted += OnLogout;
IsBroadcasting = true;
Status = "Capture en cours ; attente du premier segment HLS.";
@ -225,6 +229,7 @@ public partial class PodcastsPageViewModel : ViewModelBase
}
finally
{
CapturePreview = null;
broadcaster.Progress -= OnProgress;
_session.LogoutCompleted -= OnLogout;
try { await broadcaster.DisposeAsync(); }

View file

@ -10,6 +10,13 @@
<StackPanel Margin="16" Spacing="12" MaxWidth="900">
<TextBlock Text="Podcasts : directs et replays" FontSize="22" />
<TextBlock Text="Lecture et emission reservees aux utilisateurs connectes. Quitter cette page arrete la capture." TextWrapping="Wrap" />
<StackPanel Spacing="8" IsVisible="{Binding IsBroadcasting}">
<TextBlock Text="Apercu local de la capture" FontWeight="SemiBold" />
<Border Background="Black" CornerRadius="4" ClipToBounds="True">
<ContentControl x:Name="CapturePreviewHost" Content="{Binding CapturePreview}" Height="300"
HorizontalContentAlignment="Stretch" VerticalContentAlignment="Stretch" />
</Border>
</StackPanel>
<StackPanel Spacing="8" IsVisible="{Binding CanBroadcast}" IsEnabled="{Binding !IsBroadcasting}">
<TextBox Text="{Binding Title}" PlaceholderText="Titre du direct" MaxLength="160" />
<Button Content="Detecter cameras et microphones" Command="{Binding DetectDevicesCommand}" />