notazof/yavsc
Watch
1
1
Fork
You've already forked yavsc
 0
Code Issues 4 Pull requests Projects 2 Releases 34 Packages Wiki Activity Actions

feat(calls): add Android LAN peer calls using Podcast capture

Browse source 
Share the LibVLC SRTP engine with Linux and feed Android MediaRecorder output through its native pipe without transcoding or HLS files. Reuse front-camera capture, preview, remote rendering and foreground service with consent-based controls and background cleanup.

Use separate RTP/RTCP ports to avoid the diagnosed Android VLC SDP crash. Add physical-device media checks and lifecycle regressions; document LAN/VPN scope and remaining separate-device validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Paul Schneider 2026-10-26 04:15:19 +00:00
commit a676ef3575
Signed by: notazof
GPG key ID: AD4B9B545FBD7D12
19 changed files with 718 additions and 105 deletions
Download patch file Download diff file Expand all files Collapse all files

1
CHANGELOG.md
View file

@ -5,6 +5,7 @@
### Added
* [PostIt.Android/Desktop] Extend LAN/VPN SRTP calls to Android 8+ using the existing front-camera/microphone Podcast capture, rotated native preview and embedded LibVLC remote video, without libwebrtc or new dependencies. Pass the live MediaRecorder pipe directly to the shared SRTP engine without HLS files or transcoding; add a call-specific foreground notification, stop on backgrounding, and allow incoming callers to configure devices before acceptance. Use separate RTP/RTCP UDP ports to avoid a diagnosed Android VLC `rtcp-mux` SDP-generation crash. Add debug-only physical camera/SRTP checks and shared control/pipe-input regressions; Internet/TURN/group calls remain outside this transport.
* [PostIt.Desktop/API] Add opt-in Linux LAN/VPN peer calls through the existing ChatHub, reusing Podcast LibVLC camera/microphone capture and H.264/AAC encoding with direct SRTP instead of HLS. Add consent, embedded local/remote video, hangup/interruption cleanup, private IPv4 validation and per-direction ephemeral keys over HTTPS-only signaling. Reject IP-masking accounts rather than silently bypassing TURN; keep this transport disabled by default and separate from the future Internet/group-call design. Test real bidirectional encrypted audio/video, wrong-key rejection, preview output, signaling and UI lifecycle; physical camera/two-machine rendering remains to validate.
* [PostIt/API] Begin Android/Linux call integration on the existing authenticated ChatHub: typed invitation, single-device consent, bounded SDP/ICE exchange and hangup; derive relay policy from account preferences and issue temporary coturn credentials. Attach the shared PostIt signaling client to the chat connection, end calls on disconnect/room departure/privacy activation, and add targeted regression tests. Calls remain disabled by default; native media adapters and Android/Linux end-to-end validation are not implemented yet.
* [PostIt] Clarify reuse of the existing ChatHub for call signaling and document the inconclusive Android native WebRTC experiment; discard the isolated laboratory without changing production chat or media behavior.

125
doc/architecture/audio-video-calls.md
View file

@ -5,9 +5,9 @@
Ce document décrit l'architecture cible et l'implémentation commencée.
La préférence de confidentialité est persistée et éditable dans le profil.
La signalisation est intégrée au chat, désactivée par défaut. Un premier
transport natif **Desktop Linux vers Desktop Linux sur LAN/VPN** est
raccordé : capture LibVLC existante et SRTP direct, sans libwebrtc ni JNI
Android. Il ne réalise pas encore la cible Internet, TURN ou groupe.
transport natif **Android et Desktop Linux sur LAN/VPN** est raccordé :
captures Podcast existantes et SRTP direct, sans libwebrtc ni nouvelle
couche JNI WebRTC. Il ne réalise pas encore la cible Internet, TURN ou groupe.
Il complète la [topologie de PostIt](postit.md) et les
[mécanismes de notification existants](notifications.md).
La cible couvre :
@ -18,24 +18,27 @@ La cible couvre :
- les appels de groupe ;
- les clients Browser, Android et Desktop.
## Première implémentation Desktop Linux : SRTP sur LAN/VPN
## Première implémentation Android et Desktop Linux : SRTP sur LAN/VPN
Cette tranche réutilise la capture caméra/microphone et les encodeurs des
Podcasts, mais **pas leur segmentation HLS ni leurs envois de fichiers** :
```text
PostIt Linux A ----- ChatHub authentifié / HTTPS ----- PostIt Linux B
PostIt A ----------- ChatHub authentifié / HTTPS ----------- PostIt B
invitation, consentement,
adresses et clés, fin
PostIt Linux A ===== H.264/AAC, MPEG-TS sur SRTP/UDP ===== PostIt Linux B
PostIt A ========== H.264/AAC, MPEG-TS sur SRTP/UDP ========== PostIt B
dans les deux sens
```
L'implémentation est dans
[LinuxLanCallMedia](../../src/PostIt/PostIt/Services/LinuxLanCallMedia.cs),
[LibVlcLanCallMedia](../../src/PostIt/PostIt/Services/LibVlcLanCallMedia.cs),
[LinuxLanCallVideo](../../src/PostIt/PostIt/Services/LinuxLanCallVideo.cs) et
[LanCallViewModel](../../src/PostIt/PostIt/ViewModels/Chat/LanCallViewModel.cs).
Android utilise
[AndroidLanCallMedia](../../src/PostIt/PostIt.Android/Services/AndroidLanCallMedia.cs)
avec les mêmes contrats et clés SRTP.
Les contrôles et les deux surfaces vidéo sont intégrés à la page du chat.
Il n'y a pas de service de salles supplémentaire, d'enregistrement,
de fichier média temporaire ou de moteur WebRTC Android ajouté.
@ -58,19 +61,23 @@ de fichier média temporaire ou de moteur WebRTC Android ajouté.
invitation, acceptation et échange de clés LAN sont refusés en HTTP.
Derrière un proxy TLS, configurer correctement le schéma HTTPS transmis
à ASP.NET Core, uniquement depuis les proxys de confiance.
2. Utiliser deux Desktop Linux avec LibVLC et ses modules V4L2, PulseAudio,
x264, AAC et RTP/SRTP installés. La version Debian 13 a été utilisée
pour les essais natifs. Les deux comptes doivent être présents dans
2. Utiliser deux clients Android 8+ ou Desktop Linux. Linux nécessite LibVLC
avec V4L2, PulseAudio, x264, AAC et RTP/SRTP installés. Android réutilise
la bibliothèque VLC embarquée pour les Podcasts et MediaRecorder pour
H.264/AAC. La version Debian 13 a été utilisée pour les essais Linux.
Les deux comptes doivent être présents dans
la même salle existante et ne pas se bloquer mutuellement.
3. Dans « Appel Linux direct (LAN/VPN) », actualiser les périphériques,
3. Dans « Appel direct (LAN/VPN) », actualiser les périphériques,
sélectionner la caméra, le microphone et une IPv4 privée locale
accessible depuis le pair. Les adresses RFC1918 et de bouclage sont
acceptées ; pas de DNS, IPv6 ou adresse publique. Le bouclage sert aux
essais sur un même hôte, pas à joindre un second poste.
4. Choisir un port UDP **pair entre 1024 et 65534**, par défaut 5004.
Autoriser l'entrée UDP sur ce port depuis le pair dans le pare-feu.
RTP/RTCP partagent ce port. Deux clients sur la même adresse doivent
utiliser des ports distincts. Aucun pare-feu n'est modifié automatiquement.
Autoriser l'entrée UDP sur ce port **et le suivant** depuis le pair dans
le pare-feu : RTP sur le port pair, RTCP sur le port impair suivant.
Les deux ports doivent être disponibles. Deux clients sur la même
adresse doivent utiliser des paires distinctes.
Aucun pare-feu n'est modifié automatiquement.
5. Saisir le pseudo exact du destinataire, confirmer la divulgation de
l'adresse LAN/VPN, puis appeler. Le destinataire choisit ses propres
périphériques et son adresse, confirme et accepte explicitement.
@ -81,6 +88,43 @@ de fichier média temporaire ou de moteur WebRTC Android ajouté.
arrêtent et libèrent les médias. Il n'y a pas de reprise automatique
de capture après reconnexion.
### Capture et cycle de vie Android
- La caméra avant est sélectionnée en priorité dans la liste ; une autre
caméra reste sélectionnable explicitement. Le microphone est obligatoire.
- Les permissions caméra, microphone et notifications sont demandées
après acceptation et réception de l'endpoint distant, avant la capture.
Un refus termine explicitement l'appel, sans tentative de capture de repli.
- Camera2, MediaRecorder et la rotation EGL sont ceux des Podcasts.
Le descripteur du pipe MPEG-TS est lu directement par LibVLC (`fd://`)
puis remultiplexé vers SRTP, **sans réencodage**, HLS ou fichier média.
Le pipe système assure une mémoire bornée ; il n'y a pas de file de
segments ni de lecture de capture par callbacks managés.
- La prévisualisation réutilise la surface EGL existante. Le retour
distant utilise la surface Android LibVLC déjà employée pour les Podcasts.
- Le service de capture partagé affiche une notification d'appel avec
l'action « Raccrocher » ; cette action ne publie aucun Podcast.
Un Podcast et un appel ne peuvent pas posséder la capture simultanément.
- Le passage de PostIt en arrière-plan termine cet appel vidéo ; revenir
dans l'application ne le redémarre pas. La sortie de page et la perte
SignalR conservent la même règle d'arrêt.
- À l'arrêt, le lecteur natif libère d'abord son entrée. Le pipe restant
est drainé pendant la finalisation de MediaRecorder, pour ne pas bloquer
son écrivain, puis caméra, microphone, lecteurs et service sont libérés.
#### Particularité VLC Android : RTCP séparé
L'essai matériel initial a identifié un SIGSEGV dans `libc.__vfprintf`,
appelé par `vlc_memstream_vprintf` puis `sdp_AddAttribute` pendant la
création de la sortie RTP. Le code VLC 3 génère l'attribut sans valeur
`rtcp-mux` avec un format nul ; cette combinaison échoue avec le binaire
Android embarqué. Ce crash est distinct du précédent essai libwebrtc/JNI.
Les appels utilisent donc une paire RTP/RTCP standard, sans `rtcp-mux`,
sur Android **et** Linux. Les médias conservent les mêmes codecs, clés et
port RTP : c'est un changement de configuration du transport, pas
l'ajout d'un moteur WebRTC ou d'une cryptographie propre au projet.
### Confidentialité et limites
- Une clé SRTP aléatoire de 128 bits et un sel de 112 bits sont générés
@ -95,7 +139,7 @@ de fichier média temporaire ou de moteur WebRTC Android ajouté.
**refusé**, y compris si la préférence change avant l'acceptation.
Il n'y a aucun repli direct contournant cette préférence.
- Pas d'ICE, de STUN, de TURN, de traversée NAT automatique, de groupe,
d'interopérabilité Browser/Android, de négociation de codecs ni
d'interopérabilité Browser, de négociation de codecs ni
d'adaptation de débit. Ce mode exige un LAN/VPN routable entre les pairs.
- H.264 à 30 images/s, cible 1 200 kbit/s ; AAC mono 48 kHz, 96 kbit/s.
Ces réglages ne constituent pas une garantie de latence ni de débit.
@ -116,12 +160,15 @@ synthétique fourni explicitement et de vrais sockets UDP de bouclage :
- refus de décodage avec une mauvaise clé, sans casser l'autre direction ;
- même échange avec la branche de prévisualisation locale activée
(sortie factice dans ce test sans fenêtre).
- entrée MPEG-TS déjà encodée à travers un pipe, sans transcodage.
Les tests de signalisation couvrent consentement, sélection d'un seul
appareil, confidentialité, HTTPS, validation des endpoints et arrêt.
Les tests Avalonia sans fenêtre couvrent l'ordre invitation/acceptation/
capture, le refus, les événements tardifs, l'interruption, les erreurs média
et l'annulation d'un démarrage natif en attente.
et l'annulation d'un démarrage natif en attente. Ils vérifient aussi les
hooks Android, le choix initial de caméra avant, la configuration lors
d'une invitation entrante et l'arrêt lors du passage en arrière-plan.
Pour exécuter les tests média, définir `POSTIT_TEST_MEDIA` avec le chemin
d'un MP4 H.264/AAC de test (640 × 480), puis :
@ -133,10 +180,45 @@ dotnet test --project src/PostIt/PostIt.Tests/PostIt.Tests.csproj \
Sans cette entrée explicite, ces tests sont ignorés, pas remplacés par un
succès fictif. Les tests de contrôle s'exécutent sans caméra.
Le test de pipe utilise `POSTIT_TEST_TS`, un MPEG-TS H.264/AAC explicite.
Une activité **Debug uniquement**,
[AndroidLanCallTestActivity](../../src/PostIt/PostIt.Tests/AndroidLanCallTestActivity.cs),
teste l'adaptateur réel sur téléphone : caméra avant/microphone, surfaces
de retour et prévisualisation attachées, deux moteurs SRTP en UDP de
bouclage, retour synthétique et rejet d'une mauvaise clé. Elle n'utilise
ni compte ni API ; elle ne valide donc pas à elle seule un appel SignalR
entre deux comptes. L'activité n'est pas compilée dans l'APK Release.
Pour la lancer sur un appareil de développement dont les permissions
ont été accordées, installer l'APK Debug et ses assemblies via
`dotnet build -t:Install`, puis copier un MPEG-TS synthétique de 40 secondes,
640 × 480 avec audio AAC, sous `files/lan-call-fixture.ts` dans les données
privées de l'application. Exécuter :
```sh
adb shell am start -S -n fr.pschneider.postit/fr.pschneider.postit.LanCallTestActivity
adb shell run-as fr.pschneider.postit cat files/lan-call-test-result.txt
# Variante : ajouter --es wrongKey true à la commande am start.
```
Le résultat ne devient `PASS` qu'après vérification des médias et nettoyage
complet. Le test ne produit aucun enregistrement de caméra.
Sur le téléphone Android 15 arm64 connecté, l'essai positif a décodé
158 trames caméra et 399 360 échantillons audio en environ dix secondes,
avec un retour audio/vidéo décodé simultanément. L'essai avec mauvaise clé
a produit zéro trame et zéro échantillon dans la direction concernée,
tout en conservant le retour. Les deux essais ont terminé leur nettoyage,
sans service de capture restant. La capture Podcast antérieure a également
été retestée sur caméra avant : quatre segments, environ 11,8 secondes,
audio AAC présent, capture en arrière-plan puis arrêt par notification.
**La caméra physique, le rendu natif visible dans deux applications, le
son audible et un appel entre deux machines restent à valider** : aucune
caméra n'était disponible sur l'hôte de développement. Les essais de
bouclage ne prouvent ni ces points ni une latence conversationnelle.
son audible et un appel entre deux machines restent à valider côté Linux** :
aucune caméra n'était disponible sur cet hôte. La capture matérielle Android
est testée, mais un appel Android ↔ Linux ou entre deux téléphones distincts
sur LAN/VPN et la qualité acoustique restent à valider. Les essais de
bouclage ne prouvent pas une latence conversationnelle.
## Choix des protocoles
@ -424,8 +506,9 @@ automatique.
### Travail restant pour la cible Internet multi-plateforme
- Android : l'approche Java/JNI libwebrtc a été écartée à la demande du
projet ; aucun adaptateur expérimental n'est conservé. Le transport
compatible avec la capture Podcast et la cible Internet reste à choisir.
projet ; aucun adaptateur libwebrtc expérimental n'est conservé.
L'adaptateur SRTP LAN réutilise désormais la capture Podcast ; la solution
compatible avec la cible Internet reste à choisir.
- Desktop Linux : le transport SRTP LAN est raccordé, mais ne remplace pas
les validations matérielles ni une solution de connectivité Internet.
- Côté partagé : relier consentement, états réels du moteur, mise en

5
src/PostIt/PostIt.Android/PlatformBootstrap.cs
View file

@ -24,6 +24,11 @@ internal static class PlatformBootstrap
Platform.CreateVideoSurface = (player, replay) => new AndroidVideoSurface(player, replay);
if (System.OperatingSystem.IsAndroidVersionAtLeast(26))
{
Platform.GetLanCallDevices = AndroidLanCallMedia.GetDevices;
Platform.CreateLanCallMedia = (address, port) =>
System.OperatingSystem.IsAndroidVersionAtLeast(26)
? new AndroidLanCallMedia(address, port)
: throw new System.PlatformNotSupportedException("Android 8 requis pour les appels LAN.");
Platform.GetPodcastCaptureDevices = AndroidPodcastCapture.GetDevices;
Platform.RequestPodcastPermissionsAsync = AndroidPodcastCapture.RequestPermissionsAsync;
Platform.CreatePodcastCaptureAsync = AndroidPodcastCapture.StartAsync;

1
src/PostIt/PostIt.Android/PostIt.Android.csproj
View file

@ -32,6 +32,7 @@
<ItemGroup Condition="'$(Configuration)' == 'Debug'">
<Compile Include="..\PostIt.Tests\AndroidPodcastCaptureTestActivity.cs" Link="Tests\AndroidPodcastCaptureTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidCameraFrameRotationTestActivity.cs" Link="Tests\AndroidCameraFrameRotationTestActivity.cs" />
<Compile Include="..\PostIt.Tests\AndroidLanCallTestActivity.cs" Link="Tests\AndroidLanCallTestActivity.cs" />
</ItemGroup>
<!-- This is an Avalonia app, not a MAUI app: exclude the .NET MAUI VS Code
extension's injected HotReload agent (references Microsoft.Maui.*).

117
src/PostIt/PostIt.Android/Services/AndroidLanCallMedia.cs Normal file
View file

@ -0,0 +1,117 @@
using System;
using System.Linq;
using System.Runtime.Versioning;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using PostIt.Services;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Android.Services;
[SupportedOSPlatform("android26.0")]
internal sealed class AndroidLanCallMedia : ILanCallMedia
{
private readonly LibVlcLanCallMedia _media;
private readonly ContentControl _local = new();
private readonly AndroidVideoSurface _remote;
private readonly CancellationTokenSource _lifetime = new();
private readonly TaskCompletionSource _captureEnded = new(TaskCreationOptions.RunContinuationsAsynchronously);
private readonly Task _receiving;
private IPodcastCapture? _capture;
private bool _disposed;
public Control LocalVideo => _local;
public Control RemoteVideo => _remote;
public LanCallEndpoint Endpoint => _media.Endpoint;
public bool HasRemoteMedia => !_disposed && _media.HasRemoteMedia;
public Task Completion { get; }
public static PodcastCaptureDevices GetDevices()
{
var devices = AndroidPodcastCapture.GetDevices();
return new(devices.Cameras.OrderByDescending(c => c.Name == "Camera avant").ToArray(),
devices.Microphones.Where(m => m.Value == "microphone").ToArray());
}
public AndroidLanCallMedia(string localAddress, int port)
{
_media = new LibVlcLanCallMedia(localAddress, port);
_remote = new AndroidVideoSurface(_media.Receiver, replay: false);
_receiving = ReceiveAsync();
Completion = ObserveAsync();
}
private async Task ReceiveAsync()
{
await _remote.Ready.WaitAsync(TimeSpan.FromSeconds(10), _lifetime.Token);
await _media.StartReceivingAsync(_lifetime.Token);
}
private async Task ObserveAsync()
{
var first = await Task.WhenAny(_media.Completion, _captureEnded.Task, _receiving);
await first;
if (first == _receiving)
{
var finished = await Task.WhenAny(_media.Completion, _captureEnded.Task);
await finished;
}
}
public async Task StartSendingAsync(LanCallEndpoint remote, string camera, string microphone, CancellationToken ct)
{
ObjectDisposedException.ThrowIf(_disposed, this);
if (_capture != null) throw new InvalidOperationException("La capture de cet appel est deja demarree.");
using var linked = CancellationTokenSource.CreateLinkedTokenSource(ct, _lifetime.Token);
_capture = await AndroidPodcastCapture.StartCallAsync(camera, microphone,
(fd, token) => _media.StartEncodedSendingAsync(remote, fd, token), _media.StopAsync, linked.Token);
_local.Content = _capture.Preview;
_ = ObserveCaptureAsync(_capture);
}
private async Task ObserveCaptureAsync(IPodcastCapture capture)
{
try
{
await capture.Completion;
if (!_disposed)
_captureEnded.TrySetException(new InvalidOperationException("Capture de l'appel arretee."));
}
catch (Exception error) { _captureEnded.TrySetException(error); }
}
public async ValueTask DisposeAsync()
{
if (_disposed) return;
_disposed = true;
_lifetime.Cancel();
var stop = _capture?.StopAsync();
try
{
if (stop != null) await stop;
}
finally
{
try
{
if (_capture != null) await _capture.DisposeAsync();
}
finally
{
_local.Content = null;
try { await _media.StopAsync(); }
finally
{
_remote.Dispose();
try { await _media.DisposeAsync(); }
finally
{
_lifetime.Dispose();
_captureEnded.TrySetResult();
}
}
}
}
}
}

112
src/PostIt/PostIt.Android/Services/AndroidPodcastCapture.cs
View file

@ -13,6 +13,7 @@ using Android.OS;
using Android.Views;
using Microsoft.Win32.SafeHandles;
using PostIt.Services;
using Stream = System.IO.Stream;
namespace PostIt.Android.Services;
@ -33,6 +34,7 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
private ParcelFileDescriptor? _readPipe;
private ParcelFileDescriptor? _writePipe;
private Task _pump = Task.CompletedTask;
private Func<Task>? _stopOutput;
private Exception? _failure;
private bool _stopped;
private bool _stopRequested;
@ -71,16 +73,26 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
}
public static async Task<IPodcastCapture> StartAsync(string camera, string? microphone, string directory, CancellationToken ct)
=> await StartCoreAsync(camera, microphone,
(stream, token) => new TransportStreamCapture(directory).RunAsync(stream, token), null, null, ct);
internal static Task<IPodcastCapture> StartCallAsync(string camera, string microphone,
Func<int, CancellationToken, Task> output, Func<Task> stopOutput, CancellationToken ct) =>
StartCoreAsync(camera, microphone, null, output, stopOutput, ct);
private static async Task<IPodcastCapture> StartCoreAsync(string camera, string? microphone,
Func<Stream, CancellationToken, Task>? output,
Func<int, CancellationToken, Task>? pipeOutput, Func<Task>? stopOutput, CancellationToken ct)
{
await RequestPermissionsAsync(microphone, ct);
var activity = MainActivity.Current ?? throw new InvalidOperationException("Activite Android indisponible.");
var service = await PodcastCaptureService.StartAsync(activity, microphone is not null, ct);
var capture = new AndroidPodcastCapture(service);
var service = await PodcastCaptureService.StartAsync(activity, microphone is not null, ct, pipeOutput != null);
var capture = new AndroidPodcastCapture(service) { _stopOutput = stopOutput };
service.Capture = capture;
try
{
ct.ThrowIfCancellationRequested();
await capture.StartNativeAsync(camera, microphone is not null, directory, ct);
await capture.StartNativeAsync(camera, microphone is not null, output, pipeOutput, ct);
return capture;
}
catch
@ -90,14 +102,18 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
}
}
private async Task StartNativeAsync(string cameraId, bool microphone, string directory, CancellationToken ct)
private async Task StartNativeAsync(string cameraId, bool microphone,
Func<Stream, CancellationToken, Task>? output,
Func<int, CancellationToken, Task>? pipeOutput, CancellationToken ct)
{
await _stopGate.WaitAsync(ct);
try { await ConfigureNativeAsync(cameraId, microphone, directory, ct); }
try { await ConfigureNativeAsync(cameraId, microphone, output, pipeOutput, ct); }
finally { _stopGate.Release(); }
}
private async Task ConfigureNativeAsync(string cameraId, bool microphone, string directory, CancellationToken ct)
private async Task ConfigureNativeAsync(string cameraId, bool microphone,
Func<Stream, CancellationToken, Task>? output,
Func<int, CancellationToken, Task>? pipeOutput, CancellationToken ct)
{
var manager = GetManager(_service);
using var characteristics = manager.GetCameraCharacteristics(cameraId);
@ -179,28 +195,34 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
using var request = _camera.CreateCaptureRequest(CameraTemplate.Record);
request.AddTarget(cameraSurface);
using var built = request.Build();
var handle = new SafeFileHandle((IntPtr)_readPipe.Fd, ownsHandle: false);
var stream = new FileStream(handle, FileAccess.Read);
_pump = Task.Run(async () =>
if (pipeOutput != null)
await pipeOutput(_readPipe.Fd, ct);
else
{
using (handle)
using (stream)
var consume = output ?? throw new InvalidOperationException("Destination de capture absente.");
var handle = new SafeFileHandle((IntPtr)_readPipe.Fd, ownsHandle: false);
var stream = new FileStream(handle, FileAccess.Read);
_pump = Task.Run(async () =>
{
try
using (handle)
using (stream)
{
await new TransportStreamCapture(directory).RunAsync(stream, CancellationToken.None);
if (_failure is not null) throw new IOException("Capture Android interrompue.", _failure);
if (!_stopped) throw new IOException("Le flux de capture Android s'est ferme prematurement.");
_completion.TrySetResult();
try
{
await consume(stream, CancellationToken.None);
if (_failure is not null) throw new IOException("Capture Android interrompue.", _failure);
if (!_stopped) throw new IOException("Le flux de capture Android s'est ferme prematurement.");
_completion.TrySetResult();
}
catch (Exception ex)
{
_completion.TrySetException(ex);
// Drain until MediaRecorder stops, so its writer cannot block Stop().
await stream.CopyToAsync(Stream.Null);
}
}
catch (Exception ex)
{
_completion.TrySetException(ex);
// Drain until MediaRecorder stops, so its writer cannot block Stop().
await stream.CopyToAsync(System.IO.Stream.Null);
}
}
});
});
}
await Task.Run(_recorder.Start, ct);
_started = true;
await _rotator.StartAsync();
@ -289,14 +311,48 @@ internal sealed class AndroidPodcastCapture : IPodcastCapture
{
if (_stopped) return;
_stopped = true;
if (_preview is not null)
await Avalonia.Threading.Dispatcher.UIThread.InvokeAsync(_preview.Dispose);
await Task.Run(StopNative);
await _pump;
try
{
if (_preview is not null)
await Avalonia.Threading.Dispatcher.UIThread.InvokeAsync(_preview.Dispose);
}
finally
{
try { await StopOutputAsync(); }
finally
{
await Task.Run(StopNative);
await _pump;
}
}
_completion.TrySetResult();
}
catch (Exception error)
{
_completion.TrySetException(error);
throw;
}
finally { _stopGate.Release(); }
}
private async Task StopOutputAsync()
{
if (_stopOutput == null || _readPipe == null) return;
try { await _stopOutput(); }
finally
{
// The native sender has released its reader. Drain so the recorder can always stop.
var handle = new SafeFileHandle((IntPtr)_readPipe.Fd, ownsHandle: false);
var stream = new FileStream(handle, FileAccess.Read);
_pump = Task.Run(async () =>
{
using (handle)
using (stream)
await stream.CopyToAsync(Stream.Null);
});
}
}
private void StopNative()
{
try

8
src/PostIt/PostIt.Android/Services/AndroidVideoSurface.cs
View file

@ -1,4 +1,5 @@
using System;
using System.Threading.Tasks;
using Avalonia.Android;
using Avalonia.Controls;
using Avalonia.Platform;
@ -18,6 +19,8 @@ internal sealed class AndroidVideoSurface(MediaPlayer player, bool replay) : Nat
private VideoView? _video;
private ReplayLayout? _layout;
private bool _disposed;
private readonly TaskCompletionSource _ready = new(TaskCreationOptions.RunContinuationsAsynchronously);
internal Task Ready => _ready.Task;
protected override IPlatformHandle CreateNativeControlCore(IPlatformHandle parent)
{
@ -25,6 +28,10 @@ internal sealed class AndroidVideoSurface(MediaPlayer player, bool replay) : Nat
?? throw new InvalidOperationException("L'activite Android est indisponible.");
var video = new VideoView(activity) { MediaPlayer = _disposed ? null : player };
_video = video;
Avalonia.Threading.Dispatcher.UIThread.Post(() =>
{
if (!_disposed) _ready.TrySetResult();
}, Avalonia.Threading.DispatcherPriority.Loaded);
if (replay && !_disposed)
{
_layout = new ReplayLayout(activity, video, player);
@ -45,6 +52,7 @@ internal sealed class AndroidVideoSurface(MediaPlayer player, bool replay) : Nat
public void Dispose()
{
_disposed = true;
_ready.TrySetCanceled();
_layout?.Stop();
if (_video is not null) _video.MediaPlayer = null;
}

14
src/PostIt/PostIt.Android/Services/PodcastCaptureService.cs
View file

@ -21,14 +21,16 @@ public sealed class PodcastCaptureService : Service
private PowerManager.WakeLock? _wakeLock;
internal AndroidPodcastCapture? Capture { get; set; }
public static async Task<PodcastCaptureService> StartAsync(Activity activity, bool microphone, CancellationToken ct)
public static async Task<PodcastCaptureService> StartAsync(Activity activity, bool microphone,
CancellationToken ct, bool call = false)
{
if (_pending is not null) throw new InvalidOperationException("Un direct Android est deja en cours.");
var pending = new TaskCompletionSource<PodcastCaptureService>(TaskCreationOptions.RunContinuationsAsynchronously);
_pending = pending;
try
{
var intent = new Intent(activity, typeof(PodcastCaptureService)).PutExtra("microphone", microphone);
var intent = new Intent(activity, typeof(PodcastCaptureService))
.PutExtra("microphone", microphone).PutExtra("call", call);
activity.StartForegroundService(intent);
return await pending.Task.WaitAsync(TimeSpan.FromSeconds(10), ct);
}
@ -65,11 +67,13 @@ public sealed class PodcastCaptureService : Service
PendingIntentFlags.UpdateCurrent | PendingIntentFlags.Immutable);
var builder = new NotificationCompat.Builder(this, Channel);
builder.SetSmallIcon(global::Android.Resource.Drawable.IcMenuCamera);
builder.SetContentTitle("Podcast en direct");
builder.SetContentText("Camera active. Toucher pour revenir au direct.");
var call = intent?.GetBooleanExtra("call", false) == true;
builder.SetContentTitle(call ? "Appel audio-video LAN/VPN" : "Podcast en direct");
builder.SetContentText(call ? "Camera et microphone actifs. Toucher pour revenir a l'appel."
: "Camera active. Toucher pour revenir au direct.");
builder.SetContentIntent(open);
builder.SetOngoing(true);
builder.AddAction(global::Android.Resource.Drawable.IcMediaPause, "Arreter et publier", stop);
builder.AddAction(global::Android.Resource.Drawable.IcMediaPause, call ? "Raccrocher" : "Arreter et publier", stop);
var notification = builder.Build();
var type = (ForegroundService)0;
if (OperatingSystem.IsAndroidVersionAtLeast(30))

158
src/PostIt/PostIt.Tests/AndroidLanCallTestActivity.cs Normal file
View file

@ -0,0 +1,158 @@
#if ANDROID
using System;
using Stopwatch = System.Diagnostics.Stopwatch;
using System.IO;
using System.IO.Pipes;
using System.Linq;
using System.Runtime.InteropServices;
using System.Runtime.Versioning;
using System.Threading;
using System.Threading.Tasks;
using Android.App;
using Android.Content;
using Android.OS;
using Avalonia.Controls;
using Avalonia.Threading;
using LibVLCSharp.Shared;
using PostIt.Android;
using PostIt.Android.Services;
using PostIt.Services;
using Path = System.IO.Path;
namespace PostIt.Tests;
/// <summary>Debug-only physical camera/SRTP test; no account, API or media recording.</summary>
[Activity(Name = "fr.pschneider.postit.LanCallTestActivity", Exported = true)]
[SupportedOSPlatform("android26.0")]
public sealed class AndroidLanCallTestActivity : Activity
{
protected override async void OnCreate(Bundle? state)
{
base.OnCreate(state);
var result = Path.Combine(FilesDir!.AbsolutePath, "lan-call-test-result.txt");
File.WriteAllText(result, "RUNNING");
try
{
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(55));
var outcome = await RunAsync(result, Intent?.GetStringExtra("wrongKey") == "true", timeout.Token);
File.WriteAllText(result, outcome);
}
catch (Exception error) { File.WriteAllText(result, "FAIL\n" + error); }
finally { Finish(); }
}
private async Task<string> RunAsync(string result, bool wrongKey, CancellationToken ct)
{
StartActivity(new Intent(this, typeof(MainActivity)).AddFlags(ActivityFlags.NewTask));
while (!MainActivity.IsForeground || (Avalonia.Application.Current as App)?.View == null)
await Task.Delay(100, ct);
await Task.Delay(1000, ct);
var fixture = Path.Combine(FilesDir!.AbsolutePath, "lan-call-fixture.ts");
if (!File.Exists(fixture)) throw new IOException("Explicit H.264/AAC MPEG-TS fixture required.");
using var pipe = new AnonymousPipeServerStream(PipeDirection.Out);
await using var peer = new LibVlcLanCallMedia("127.0.0.1", 35102, headless: true);
using var decoded = new DecodedFrames(peer.Receiver);
AndroidLanCallMedia? call = null;
Task? pump = null;
using var sending = CancellationTokenSource.CreateLinkedTokenSource(ct);
var source = File.OpenRead(fixture);
var app = (App)Avalonia.Application.Current!;
var navigation = app.View!.FindControl<NavigationPage>("NavRoot")
?? throw new InvalidOperationException("Main navigation unavailable.");
var attached = false;
try
{
await peer.StartReceivingAsync(ct);
await Dispatcher.UIThread.InvokeAsync(async () =>
{
call = new AndroidLanCallMedia("127.0.0.1", 35100);
var grid = new Grid { RowDefinitions = new RowDefinitions("*,*") };
grid.Children.Add(call.LocalVideo);
Grid.SetRow(call.RemoteVideo, 1);
grid.Children.Add(call.RemoteVideo);
await navigation.PushAsync(new ContentPage { Content = grid });
attached = true;
});
if (call == null) throw new InvalidOperationException("Call UI not created.");
File.WriteAllText(result, "RUNNING\nstage=return source pump");
pump = Task.Run(async () =>
{
var buffer = new byte[1316];
while (await source.ReadAsync(buffer, sending.Token) is var count && count > 0)
{
await pipe.WriteAsync(buffer.AsMemory(0, count), sending.Token);
await Task.Delay(4, sending.Token);
}
}, ct);
File.WriteAllText(result, "RUNNING\nstage=return SRTP sender");
await peer.StartEncodedSendingAsync(call.Endpoint,
pipe.ClientSafePipeHandle.DangerousGetHandle().ToInt32(), ct);
File.WriteAllText(result, "RUNNING\nstage=enumerating cameras");
var devices = AndroidLanCallMedia.GetDevices();
var front = devices.Cameras.FirstOrDefault(c => c.Name == "Camera avant")
?? throw new InvalidOperationException("Front camera required; no rear-camera fallback.");
var endpoint = wrongKey ? peer.Endpoint with { Key = new string('0', 32) } : peer.Endpoint;
File.WriteAllText(result, "RUNNING\nstage=front camera and SRTP");
await Dispatcher.UIThread.InvokeAsync(() => call.StartSendingAsync(endpoint, front.Value, "microphone", ct));
File.WriteAllText(result, "RUNNING\nstage=receiving camera and return media");
var elapsed = Stopwatch.StartNew();
while (elapsed.Elapsed < TimeSpan.FromSeconds(wrongKey ? 10 : 25))
{
if (call.Completion.IsCompleted) await call.Completion;
if (peer.Completion.IsCompleted) await peer.Completion;
if (!wrongKey && elapsed.Elapsed >= TimeSpan.FromSeconds(10) &&
decoded.Frames >= 100 && decoded.Samples > 144000 && call.HasRemoteMedia) break;
await Task.Delay(100, ct);
}
if (!call.HasRemoteMedia) throw new IOException("Android did not decode return audio/video.");
if (wrongKey ? decoded.Frames != 0 || decoded.Samples != 0
: decoded.Frames < 100 || decoded.Samples <= 144000)
throw new IOException($"Unexpected camera decode: frames={decoded.Frames}, samples={decoded.Samples}.");
return $"PASS\nfrontCamera=true\nwrongKey={wrongKey}\n" +
$"cameraFrames={decoded.Frames}\ncameraAudioSamples={decoded.Samples}\nelapsedMs={elapsed.ElapsedMilliseconds}\nreturnAudioVideo=true\n" +
"transport=SRTP/UDP loopback\npreview=attached\nsource=camera+microphone,return=explicit fixture\ncleanup=complete";
}
finally
{
sending.Cancel();
File.WriteAllText(result, "RUNNING\nstage=stop return source");
if (pump != null)
{
try { await pump; }
catch (System.OperationCanceledException) when (sending.IsCancellationRequested) { }
}
File.WriteAllText(result, "RUNNING\nstage=dispose call");
if (call != null) await Dispatcher.UIThread.InvokeAsync(async () => await call.DisposeAsync());
File.WriteAllText(result, "RUNNING\nstage=stop return peer");
await peer.StopAsync();
source.Dispose();
File.WriteAllText(result, "RUNNING\nstage=detach test page");
if (attached) await Dispatcher.UIThread.InvokeAsync(async () => await navigation.PopAsync());
}
}
private sealed class DecodedFrames : IDisposable
{
private readonly MediaPlayer _player;
private readonly IntPtr _buffer = Marshal.AllocHGlobal(640 * 640 * 4);
private int _frames;
private long _samples;
public int Frames => Volatile.Read(ref _frames);
public long Samples => Interlocked.Read(ref _samples);
public DecodedFrames(MediaPlayer player)
{
_player = player;
player.SetVideoFormat("RV32", 640, 640, 640 * 4);
player.SetVideoCallbacks((_, planes) =>
{
Marshal.WriteIntPtr(planes, _buffer);
return IntPtr.Zero;
}, null, (_, _) => Interlocked.Increment(ref _frames));
player.SetAudioFormat("S16N", 48000, 1);
player.SetAudioCallbacks((_, _, count, _) => Interlocked.Add(ref _samples, count),
null, null, null, null);
}
public void Dispose() { _player.Stop(); Marshal.FreeHGlobal(_buffer); }
}
}
#endif

51
src/PostIt/PostIt.Tests/LanCallViewModelTests.cs
View file

@ -105,6 +105,57 @@ public sealed class LanCallViewModelTests
Assert.False(vm.Busy);
}
[AvaloniaFact]
public async Task Platform_devices_and_factory_are_used_and_backgrounding_ends_capture()
{
var previousDevices = Platform.GetLanCallDevices;
var previousFactory = Platform.CreateLanCallMedia;
var signaling = new Signaling();
var media = new FakeMedia();
var creations = 0;
try
{
Platform.GetLanCallDevices = () => new(
[new("Camera avant", "front"), new("Camera arriere", "rear")],
[new("Sans microphone", ""), new("Microphone Android", "microphone")]);
Platform.CreateLanCallMedia = (_, _) => { creations++; return media; };
await using var vm = new LanCallViewModel(signaling);
vm.RefreshDevicesCommand.Execute(null);
Assert.Equal("front", vm.Camera?.Value);
Assert.Equal("microphone", vm.Microphone?.Value);
Assert.Single(vm.Microphones);
Assert.Equal(0, creations);
signaling.Invite(vm, signaling.Call);
await WaitAsync(() => vm.Incoming);
Assert.True(vm.CanConfigure);
vm.RefreshDevicesCommand.Execute(null);
Assert.Equal("front", vm.Camera?.Value);
vm.Address = "127.0.0.1";
if (!vm.Addresses.Contains(vm.Address)) vm.Addresses.Add(vm.Address);
vm.AllowAddressDisclosure = true;
vm.Room = "General";
vm.Target = "bob";
await vm.AcceptCommand.ExecuteAsync(null);
await WaitAsync(() => vm.LocalVideo != null);
Assert.Equal(1, creations);
Assert.False(vm.CanConfigure);
signaling.RemoteEndpoint(signaling.Call.Id);
await WaitAsync(() => media.Sending);
Platform.SuspendPlayback();
await WaitAsync(() => media.Disposed);
Assert.False(vm.Busy);
Assert.Contains(signaling.Call.Id, signaling.EndedCalls);
Platform.ResumePlayback();
await Task.Delay(30, TestContext.Current.CancellationToken);
Assert.Equal(1, creations);
}
finally
{
Platform.GetLanCallDevices = previousDevices;
Platform.CreateLanCallMedia = previousFactory;
}
}
[AvaloniaFact]
public async Task Hangup_cancels_pending_native_start_without_waiting_for_startup_timeout()
{

85
src/PostIt/PostIt.Tests/LinuxLanCallMediaTests.cs
View file

@ -1,6 +1,7 @@
using System.Diagnostics;
using System.Net;
using System.Net.Sockets;
using System.IO.Pipes;
using System.Runtime.InteropServices;
using LibVLCSharp.Shared;
using PostIt.Services;
@ -11,6 +12,76 @@ public sealed class LinuxLanCallMediaTests(ITestOutputHelper output)
{
public static bool HasFixture => OperatingSystem.IsLinux() &&
File.Exists(Environment.GetEnvironmentVariable("POSTIT_TEST_MEDIA"));
public static bool HasTsFixture => OperatingSystem.IsLinux() &&
File.Exists(Environment.GetEnvironmentVariable("POSTIT_TEST_TS"));
[Fact(Skip = "Requires Linux LibVLC.", SkipUnless = nameof(HasFixture))]
public async Task Ports_are_released_and_credentials_regenerated_after_disposal()
{
var port = FreePort();
var first = new LibVlcLanCallMedia("127.0.0.1", port, headless: true);
var endpoint = first.Endpoint;
await first.StartReceivingAsync(TestContext.Current.CancellationToken);
await first.DisposeAsync();
await first.DisposeAsync();
await using var next = new LibVlcLanCallMedia("127.0.0.1", port, headless: true);
Assert.NotEqual(endpoint.Key, next.Endpoint.Key);
Assert.NotEqual(endpoint.Salt, next.Endpoint.Salt);
await next.StartReceivingAsync(TestContext.Current.CancellationToken);
}
[Fact(Skip = "Requires Linux LibVLC.", SkipUnless = nameof(HasFixture))]
public void Occupied_rtcp_port_is_rejected_before_capture_starts()
{
var port = FreePort();
using var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
socket.Bind(new IPEndPoint(IPAddress.Loopback, port + 1));
var error = Assert.Throws<SocketException>(() => new LibVlcLanCallMedia("127.0.0.1", port, headless: true));
Assert.Equal(SocketError.AddressAlreadyInUse, error.SocketErrorCode);
}
[Fact(Skip = "Set POSTIT_TEST_TS to an H.264/AAC MPEG-TS file.", SkipUnless = nameof(HasTsFixture))]
public async Task Already_encoded_capture_is_sent_over_srtp_without_transcoding()
{
var ct = TestContext.Current.CancellationToken;
var firstPort = FreePort();
await using var sender = new LibVlcLanCallMedia("127.0.0.1", firstPort, headless: true);
await using var receiver = new LibVlcLanCallMedia("127.0.0.1", FreePort(firstPort), headless: true);
using var decoded = new DecodedVideo(receiver.Receiver);
using var pipe = new AnonymousPipeServerStream(PipeDirection.Out);
await using var source = File.OpenRead(Environment.GetEnvironmentVariable("POSTIT_TEST_TS")!);
await receiver.StartReceivingAsync(ct);
using var lifetime = CancellationTokenSource.CreateLinkedTokenSource(ct);
var pump = Task.Run(async () =>
{
var buffer = new byte[1316];
while (await source.ReadAsync(buffer, lifetime.Token) is var count && count > 0)
{
await pipe.WriteAsync(buffer.AsMemory(0, count), lifetime.Token);
await Task.Delay(4, lifetime.Token);
}
}, ct);
try
{
await sender.StartEncodedSendingAsync(receiver.Endpoint,
pipe.ClientSafePipeHandle.DangerousGetHandle().ToInt32(), ct);
var elapsed = Stopwatch.StartNew();
while (!HasMedia(receiver, decoded) && elapsed.Elapsed < TimeSpan.FromSeconds(20))
{
if (sender.Completion.IsCompleted) await sender.Completion;
if (receiver.Completion.IsCompleted) await receiver.Completion;
await Task.Delay(100, ct);
}
Assert.True(HasMedia(receiver, decoded));
}
finally
{
lifetime.Cancel();
try { await pump; }
catch (OperationCanceledException) when (lifetime.IsCancellationRequested) { }
await sender.StopAsync();
}
}
[Theory(Skip = "Set POSTIT_TEST_MEDIA to an audio/video file; requires Linux LibVLC SRTP and H.264/AAC encoders.",
SkipUnless = nameof(HasFixture))]
@ -22,8 +93,8 @@ public sealed class LinuxLanCallMediaTests(ITestOutputHelper output)
var ct = TestContext.Current.CancellationToken;
var firstPort = FreePort();
var secondPort = FreePort(firstPort);
await using var first = new LinuxLanCallMedia("127.0.0.1", firstPort, headless: true);
await using var second = new LinuxLanCallMedia("127.0.0.1", secondPort, headless: true);
await using var first = new LibVlcLanCallMedia("127.0.0.1", firstPort, headless: true);
await using var second = new LibVlcLanCallMedia("127.0.0.1", secondPort, headless: true);
using var firstVideo = new DecodedVideo(first.Receiver);
using var secondVideo = new DecodedVideo(second.Receiver);
await first.StartReceivingAsync(ct);
@ -68,7 +139,7 @@ public sealed class LinuxLanCallMediaTests(ITestOutputHelper output)
}
}
private static bool HasMedia(LinuxLanCallMedia peer, DecodedVideo video) =>
private static bool HasMedia(LibVlcLanCallMedia peer, DecodedVideo video) =>
peer.Statistics.DecodedAudio > 0 && video.AudioSamples > 4800 && video.Frames >= 10;
private static int FreePort(int except = 0)
@ -77,7 +148,13 @@ public sealed class LinuxLanCallMediaTests(ITestOutputHelper output)
{
if (port == except) continue;
using var socket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
try { socket.Bind(new IPEndPoint(IPAddress.Loopback, port)); return port; }
using var rtcp = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
try
{
socket.Bind(new IPEndPoint(IPAddress.Loopback, port));
rtcp.Bind(new IPEndPoint(IPAddress.Loopback, port + 1));
return port;
}
catch (SocketException ex) when (ex.SocketErrorCode == SocketError.AddressAlreadyInUse) { }
}
throw new IOException("No free RTP port available.");

17
src/PostIt/PostIt/Services/ILanCallMedia.cs Normal file
View file

@ -0,0 +1,17 @@
using System;
using System.Threading;
using System.Threading.Tasks;
using Avalonia.Controls;
using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public interface ILanCallMedia : IAsyncDisposable
{
Control LocalVideo { get; }
Control RemoteVideo { get; }
LanCallEndpoint Endpoint { get; }
Task Completion { get; }
bool HasRemoteMedia { get; }
Task StartSendingAsync(LanCallEndpoint remote, string camera, string microphone, CancellationToken ct);
}

2
src/PostIt/PostIt/Services/IPodcastCapture.cs
View file

@ -9,7 +9,7 @@ public sealed record CaptureDevice(string Name, string Value);
public sealed record PodcastCaptureDevices(
IReadOnlyList<CaptureDevice> Cameras, IReadOnlyList<CaptureDevice> Microphones);
/// <summary>A running encoder publishing closed segments and an atomic index.m3u8.</summary>
/// <summary>A running camera/microphone encoder with an optional native preview.</summary>
public interface IPodcastCapture : IAsyncDisposable
{
Task Completion { get; }

42
src/PostIt/PostIt/Services/LinuxLanCallMedia.cs → src/PostIt/PostIt/Services/LibVlcLanCallMedia.cs
View file

@ -9,7 +9,7 @@ using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public sealed class LinuxLanCallMedia : IAsyncDisposable
public sealed class LibVlcLanCallMedia : IAsyncDisposable
{
private readonly LibVLC _senderVlc;
private readonly LibVLC _receiverVlc;
@ -28,16 +28,20 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
public MediaStats Statistics => _receiverMedia.Statistics;
public bool HasRemoteMedia => !_disposed && Statistics.DecodedAudio > 0 && Statistics.DecodedVideo > 0;
public LinuxLanCallMedia(string localAddress, int port, bool headless = false)
public LibVlcLanCallMedia(string localAddress, int port, bool headless = false)
{
if (!OperatingSystem.IsLinux())
throw new PlatformNotSupportedException("Les appels SRTP LAN sont disponibles sur Desktop Linux.");
if (!OperatingSystem.IsLinux() && !OperatingSystem.IsAndroidVersionAtLeast(26))
throw new PlatformNotSupportedException("Les appels SRTP LAN requierent Linux ou Android 8+.");
Endpoint = new(localAddress, port, Convert.ToHexString(RandomNumberGenerator.GetBytes(16)),
Convert.ToHexString(RandomNumberGenerator.GetBytes(14)));
if (!Endpoint.IsValid())
throw new ArgumentException("Adresse IPv4 privee et port UDP pair (1024-65534) requis.");
using (var reservation = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
using (var rtcp = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp))
{
reservation.Bind(new IPEndPoint(IPAddress.Parse(localAddress), port));
rtcp.Bind(new IPEndPoint(IPAddress.Parse(localAddress), port + 1));
}
Core.Initialize();
_senderVlc = headless ? CameraStreamer.CreateHeadlessLibVlc()
: new LibVLC("--avcodec-hw=none", "--aout=dummy", "--intf=dummy");
@ -49,7 +53,7 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
{
_receiverMedia = new Media(_receiverVlc, $"rtp://@{localAddress}:{port}", FromType.FromLocation);
_receiverMedia.AddOption(":network-caching=150");
_receiverMedia.AddOption(":rtcp-port=0");
_receiverMedia.AddOption(":rtcp-port=" + (port + 1));
_receiverMedia.AddOption(":srtp-key=" + Endpoint.Key);
_receiverMedia.AddOption(":srtp-salt=" + Endpoint.Salt);
Receiver = new MediaPlayer(_receiverVlc);
@ -94,7 +98,7 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
StartSendingAsync(remote, vlc => new CameraStreamer(vlc).CreateCaptureMedia(camera, microphone), true, ct);
internal async Task StartSendingAsync(LanCallEndpoint remote, Func<LibVLC, Media> source,
bool preview, CancellationToken ct)
bool preview, CancellationToken ct, bool transcode = true)
{
if (!remote.IsValid())
throw new ArgumentException("Destination SRTP LAN invalide.", nameof(remote));
@ -107,7 +111,7 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
if (_stop != null || _senderMedia != null)
throw new InvalidOperationException("La capture de cet appel est deja demarree ou arretee.");
_senderMedia = source(_senderVlc);
ConfigureOutput(_senderMedia, remote, preview);
ConfigureOutput(_senderMedia, remote, preview, transcode);
ct.ThrowIfCancellationRequested();
if (!Sender.Play(_senderMedia))
throw new InvalidOperationException("Capture SRTP impossible. Verifiez camera, microphone et encodeurs H.264/AAC.");
@ -115,13 +119,27 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
finally { _gate.Release(); }
}
internal static void ConfigureOutput(Media media, LanCallEndpoint remote, bool preview)
public Task StartEncodedSendingAsync(LanCallEndpoint remote, int fileDescriptor, CancellationToken ct)
{
if (fileDescriptor < 0) throw new ArgumentOutOfRangeException(nameof(fileDescriptor));
return StartSendingAsync(remote, vlc =>
{
var media = new Media(vlc, $"fd://{fileDescriptor}", FromType.FromLocation);
media.AddOption(":demux=ts");
media.AddOption(":live-caching=100");
media.AddOption(":clock-jitter=0");
return media;
}, false, ct, transcode: false);
}
internal static void ConfigureOutput(Media media, LanCallEndpoint remote, bool preview, bool transcode = true)
{
if (!remote.IsValid()) throw new ArgumentException("Destination SRTP invalide.");
// Same capture/encoders as Podcasts, but no HLS files, segmentation or upload.
var encoding = "transcode{vcodec=h264,venc=x264{preset=ultrafast,tune=zerolatency,keyint=30,bframes=0},"
+ "vb=1200,fps=30,acodec=mp4a,ab=96,channels=1,samplerate=48000}:"
+ $"rtp{{dst={remote.Address},port={remote.Port},mux=ts,rtcp-mux,"
var encoding = (transcode ? "transcode{vcodec=h264,venc=x264{preset=ultrafast,tune=zerolatency,keyint=30,bframes=0},"
+ "vb=1200,fps=30,acodec=mp4a,ab=96,channels=1,samplerate=48000}:" : "")
// Android LibVLC 3 crashes formatting the valueless rtcp-mux SDP attribute.
+ $"rtp{{dst={remote.Address},port={remote.Port},mux=ts,"
+ $"key={remote.Key},salt={remote.Salt},caching=50}}";
media.AddOption(":sout=#" + (preview
? $"duplicate{{dst=display{{noaudio}},dst=\"{encoding}\"}}" : encoding));
@ -142,7 +160,7 @@ public sealed class LinuxLanCallMedia : IAsyncDisposable
finally { Receiver.Stop(); }
});
internal async Task StopAsync()
public async Task StopAsync()
{
await _gate.WaitAsync();
try

14
src/PostIt/PostIt/Services/LinuxLanCallVideo.cs
View file

@ -8,19 +8,9 @@ using Yavsc.Abstract.Chat.Calls;
namespace PostIt.Services;
public interface ILanCallMedia : IAsyncDisposable
{
Control LocalVideo { get; }
Control RemoteVideo { get; }
LanCallEndpoint Endpoint { get; }
Task Completion { get; }
bool HasRemoteMedia { get; }
Task StartSendingAsync(LanCallEndpoint remote, string camera, string microphone, CancellationToken ct);
}
public sealed class LinuxLanCallVideo : ILanCallMedia
{
private readonly LinuxLanCallMedia _media;
private readonly LibVlcLanCallMedia _media;
private readonly VideoView _local;
private readonly VideoView _remote;
private readonly TaskCompletionSource _localReady = new(TaskCreationOptions.RunContinuationsAsynchronously);
@ -38,7 +28,7 @@ public sealed class LinuxLanCallVideo : ILanCallMedia
public LinuxLanCallVideo(string localAddress, int port)
{
_media = new LinuxLanCallMedia(localAddress, port);
_media = new LibVlcLanCallMedia(localAddress, port);
_local = new VideoView { MediaPlayer = _media.Sender, MinHeight = 140 };
_remote = new VideoView { MediaPlayer = _media.Receiver, MinHeight = 260 };
_local.LayoutUpdated += OnLocalLayout;

2
src/PostIt/PostIt/Services/Platform.cs
View file

@ -12,6 +12,8 @@ namespace PostIt.Services;
/// </summary>
public static class Platform
{
public static Func<PodcastCaptureDevices>? GetLanCallDevices { get; set; }
public static Func<string, int, ILanCallMedia>? CreateLanCallMedia { get; set; }
public static Func<PodcastCaptureDevices>? GetPodcastCaptureDevices { get; set; }
public static Func<string?, CancellationToken, Task>? RequestPodcastPermissionsAsync { get; set; }
public static Func<string, string?, string, CancellationToken, Task<IPodcastCapture>>? CreatePodcastCaptureAsync { get; set; }

3
src/PostIt/PostIt/ViewModels/Chat/ChatViewModel.cs
View file

@ -39,7 +39,8 @@ namespace PostIt.ViewModels.Chat
public CallSignalingClient? Calls { get; private set; }
public bool SupportsLanCalls => OperatingSystem.IsLinux() && !OperatingSystem.IsAndroid();
public bool SupportsLanCalls => Platform.CreateLanCallMedia != null ||
(OperatingSystem.IsLinux() && !OperatingSystem.IsAndroid());
[ObservableProperty]
public partial LanCallViewModel? LanCall { get; set; }

45
src/PostIt/PostIt/ViewModels/Chat/LanCallViewModel.cs
View file

@ -43,22 +43,27 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
[ObservableProperty] public partial bool AllowAddressDisclosure { get; set; }
[ObservableProperty] public partial string Target { get; set; } = "";
[ObservableProperty] public partial string Room { get; set; } = "";
[ObservableProperty] public partial string Status { get; set; } = "Appel Linux LAN/VPN : aucun appel.";
[ObservableProperty] public partial bool Incoming { get; set; }
[ObservableProperty] public partial bool Busy { get; set; }
[ObservableProperty] public partial string Status { get; set; } = "Appel LAN/VPN : aucun appel.";
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
public partial bool Incoming { get; set; }
[ObservableProperty, NotifyPropertyChangedFor(nameof(CanConfigure))]
public partial bool Busy { get; set; }
[ObservableProperty] public partial bool Preparing { get; set; }
[ObservableProperty] public partial Control? LocalVideo { get; set; }
[ObservableProperty] public partial Control? RemoteVideo { get; set; }
public bool CanConfigure => !Busy || Incoming;
public LanCallViewModel(ILanCallSignaling signaling, Func<string, int, ILanCallMedia>? createMedia = null)
{
_signaling = signaling;
_createMedia = createMedia ?? ((address, port) => new LinuxLanCallVideo(address, port));
_createMedia = createMedia ?? Platform.CreateLanCallMedia ??
((address, port) => new LinuxLanCallVideo(address, port));
signaling.Invitation += OnInvitation;
signaling.Accepted += OnAccepted;
signaling.Signal += OnSignal;
signaling.Ended += OnEnded;
signaling.Interrupted += OnInterrupted;
Platform.PlaybackSuspending += OnSuspending;
}
[RelayCommand]
@ -66,7 +71,7 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
{
try
{
if (Busy) throw new InvalidOperationException("Terminez l'appel avant de changer les peripheriques.");
if (!CanConfigure) throw new InvalidOperationException("Terminez l'appel avant de changer les peripheriques.");
Addresses.Clear();
foreach (var address in NetworkInterface.GetAllNetworkInterfaces()
.Where(n => n.OperationalStatus == OperationalStatus.Up)
@ -77,14 +82,26 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
Address = Addresses.FirstOrDefault();
Cameras.Clear();
Microphones.Clear();
LibVLCSharp.Shared.Core.Initialize();
using var vlc = CameraStreamer.CreatePreferredLibVlc();
var streamer = new CameraStreamer(vlc);
foreach (var device in streamer.GetCaptureDevices()) Cameras.Add(new(device.Key, device.Value));
foreach (var device in streamer.GetAudioCaptureDevices()) Microphones.Add(new(device.Key, device.Value));
if (Platform.GetLanCallDevices is { } getDevices)
{
var devices = getDevices();
foreach (var device in devices.Cameras) Cameras.Add(device);
foreach (var device in devices.Microphones.Where(d => !string.IsNullOrEmpty(d.Value)))
Microphones.Add(device);
}
else
{
LibVLCSharp.Shared.Core.Initialize();
using var vlc = CameraStreamer.CreatePreferredLibVlc();
var streamer = new CameraStreamer(vlc);
foreach (var device in streamer.GetCaptureDevices()) Cameras.Add(new(device.Key, device.Value));
foreach (var device in streamer.GetAudioCaptureDevices()) Microphones.Add(new(device.Key, device.Value));
}
Camera = Cameras.FirstOrDefault();
Microphone = Microphones.FirstOrDefault();
Status = Cameras.Count == 0 ? "Aucune camera Linux detectee. Branchez une camera puis actualisez."
Status = Cameras.Count == 0 ? "Aucune camera detectee. Verifiez les peripheriques puis actualisez."
: Incoming && _call != null
? $"{_call.Caller} propose un appel dans {_call.RoomName}. Selectionnez vos peripheriques puis acceptez ou refusez."
: "Selectionnez le pair, l'adresse LAN/VPN et les peripheriques. Utilisez un casque.";
}
catch (Exception error) { Status = "Peripheriques indisponibles : " + error.Message; }
@ -141,7 +158,7 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
if (call.Transport != CallTransports.LanSrtp)
{
await _signaling.EndAsync(call.Id, _lifetime.Token);
Status = "Invitation WebRTC non prise en charge par le mode Linux LAN.";
Status = "Invitation WebRTC non prise en charge par le mode LAN.";
return;
}
if (_call != null) throw new InvalidOperationException("Invitation concurrente inattendue.");
@ -192,6 +209,9 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
private void OnInterrupted() => QueueStop(null,
() => StopAsync("Connexion SignalR interrompue ; medias arretes.", false));
private void OnSuspending() => QueueStop(null,
() => StopAsync("Application en arriere-plan ; appel termine.", true));
private void QueueStop(Guid? id, Func<Task> action) => Dispatcher.UIThread.Post(async () =>
{
if (id == null || _call?.Id == id) _callLifetime?.Cancel();
@ -303,6 +323,7 @@ public partial class LanCallViewModel : ObservableObject, IAsyncDisposable
_signaling.Signal -= OnSignal;
_signaling.Ended -= OnEnded;
_signaling.Interrupted -= OnInterrupted;
Platform.PlaybackSuspending -= OnSuspending;
try { await StopAsync("Appel arrete.", true); }
finally { _lifetime.Dispose(); }
}

21
src/PostIt/PostIt/Views/Chat/ChatPage.axaml
View file

@ -103,12 +103,13 @@
Content="Envoyer" />
</Grid>
<Expander Grid.Row="3" Header="Appel Linux direct (LAN/VPN)" IsVisible="{Binding SupportsLanCalls}" IsExpanded="True"
<Expander Grid.Row="3" Header="Appel direct (LAN/VPN)" IsVisible="{Binding SupportsLanCalls}" IsExpanded="True"
IsEnabled="{Binding IsConnected}" Margin="0,0,0,8">
<ScrollViewer MaxHeight="520">
<StackPanel DataContext="{Binding LanCall}" x:DataType="vm:LanCallViewModel" Spacing="8">
<TextBlock Text="Deux Desktop Linux sur LAN/VPN. SRTP chiffre les medias ; le pair connait votre IP. Sans TURN ni traversee NAT. Casque recommande (pas d'annulation d'echo)."
<TextBlock Text="Android et Desktop Linux sur LAN/VPN. SRTP chiffre les medias ; le pair connait votre IP. Ouvrir le port UDP choisi et le suivant. Sans TURN ni traversee NAT. Casque recommande (pas d'annulation d'echo)."
TextWrapping="Wrap" />
<WrapPanel IsEnabled="{Binding !Busy}">
<WrapPanel IsEnabled="{Binding CanConfigure}" IsVisible="{Binding !RemoteVideo}">
<Button Content="Actualiser les peripheriques" Command="{Binding RefreshDevicesCommand}" Margin="0,0,8,4" />
<ComboBox Width="160" ItemsSource="{Binding Addresses}" SelectedItem="{Binding Address}"
PlaceholderText="Adresse locale LAN/VPN" Margin="0,0,8,4" />
@ -132,18 +133,20 @@
<TextBox Width="170" Text="{Binding Target}" PlaceholderText="Pseudo du pair" Margin="0,0,8,4" />
</WrapPanel>
<CheckBox Content="Je consens a partager mon adresse LAN/VPN avec le pair."
IsChecked="{Binding AllowAddressDisclosure}" />
<StackPanel Orientation="Horizontal" Spacing="8">
<Button Content="Appeler dans la salle active" Command="{Binding InviteCommand}" IsEnabled="{Binding !Busy}" />
<Button Content="Accepter" Command="{Binding AcceptCommand}" IsEnabled="{Binding Incoming}" />
<Button Content="Refuser / Raccrocher" Command="{Binding HangupCommand}" IsEnabled="{Binding Busy}" />
</StackPanel>
IsChecked="{Binding AllowAddressDisclosure}" IsEnabled="{Binding CanConfigure}"
IsVisible="{Binding !RemoteVideo}" />
<WrapPanel>
<Button Content="Appeler dans la salle active" Command="{Binding InviteCommand}" IsEnabled="{Binding !Busy}" Margin="0,0,8,4" />
<Button Content="Accepter" Command="{Binding AcceptCommand}" IsEnabled="{Binding Incoming}" Margin="0,0,8,4" />
<Button Content="Refuser / Raccrocher" Command="{Binding HangupCommand}" IsEnabled="{Binding Busy}" Margin="0,0,8,4" />
</WrapPanel>
<TextBlock Text="{Binding Status}" TextWrapping="Wrap" />
<Grid ColumnDefinitions="*,2*" Height="280" IsVisible="{Binding !!RemoteVideo}">
<ContentControl Content="{Binding LocalVideo}" Margin="0,0,8,0" />
<ContentControl Grid.Column="1" Content="{Binding RemoteVideo}" />
</Grid>
</StackPanel>
</ScrollViewer>
</Expander>
<postitControls:StatusBar