David Woodhouse [Wed, 7 Mar 2018 20:32:03 +0000 (20:32 +0000)]
More audio cleanup
Fix up lifetime handling on the appsrc/appsink and uninstall the callbacks
when we're done.
Also fix up the timer handling for empty outbound packets. When the system
was slow we were sending 'muted' RT packets in between the audio frames,
so set the timers *only* when we're actually muted. If we fail to send
audio at other times, let it just do a red mic like it should.
David Woodhouse [Wed, 7 Mar 2018 16:33:38 +0000 (16:33 +0000)]
Slight cleanup for call media handling
The call is connecting before do_join_chat() is ever called, which means we
may miss some of the early state transitions. Try to cope with that.
We probably want to drive the call connection directly from chat code in
the end, but this will do for now.
Also decouple the media setup and connection; the call window in Pidgin
should now say 'Calling...' and then 'Call in progress' once it's connected.
Hopefully this helps to deal with some of the issues with slow call setup.
I *was* able to reproduce the ever-slower call connection but now I can't.
Not sure if that's just coincidence...
David Woodhouse [Wed, 7 Mar 2018 12:31:41 +0000 (12:31 +0000)]
Clean up fetch_presences() idle handling
Properly avoid re-running the function when it's already scheduled, and
ensure we cancel it when we need to too. Also ensure that disposed contacts
are removed from the 'contacts_needed' list.
Michael Gruenewald [Mon, 12 Feb 2018 15:05:31 +0000 (16:05 +0100)]
Only consider active members for mentions
Inactive members are effectively not part of the chat anymore and
shouldn't be considered for mentioning. This is especially helpful when
a member changed their profile (e.g., mail, domain, whatnot) but not
their display name.
Michael Gruenewald [Fri, 2 Feb 2018 14:39:56 +0000 (15:39 +0100)]
Prevent double-replacements in mentions
Outgoing mentions replace a name with a reference to the Chime user.
Sometimes you have two users with the same name in a chat room, or two
people where one name is a prefix of another one. This could end up with
something like `<@id1|<@id2|Last, First>>`, which is broken.
This change makes sure to only replace at word borders and not if a `|`
is in front. This isn't perfect, but better than before.
David Woodhouse [Thu, 22 Feb 2018 09:20:01 +0000 (09:20 +0000)]
Merge branch 'audio'.
I've been using it for every call I've made this week. There are still a
few things to be cleaned up, but I suppose I should concede it's time it
was merged into the master branch for others to test and heckle.
This ought to now work with the Ubuntu 16.04 packages of GStreamer,
FarStream, etc.
David Woodhouse [Thu, 22 Feb 2018 08:33:14 +0000 (08:33 +0000)]
Add Chime-specific RTP payload handling
RFC7587 defines the clock-rate for the OPUS payload format to be 48000.
This is painful since we're really quite like to be able to transpose
timestamps between the RTP and on-the-wire Chime protocol (which uses
16000) without conversion. It was OK having to multiply by three for the
appsrc, but dividing by three in the appsink would have overflow issues
and we'd have to reconstitute the high bits somehow.
By defining our own payload format we can work around that, and also
force it to encode only a single channel. That one might well be a bug
in the existing gstopuspay; if the output caps include 'stereo=0' it
*isn't* enforcing 'channels=1' on its input caps.
Both of the above could probably be fixed up in the OPUS payloaders
upstream, and once this code is public we can have a look at doing that.
For now though, having our own CHIME payload helps it work on Farstream
0.2.7 on Ubuntu 16.04, which can't cope with payloaders that have
multiple names (OPUS, X-GST-OPUS-DRAFT-SPITTKA-00). So it'll do for now
until we can get more traction on upstreaming the required fixes.
David Woodhouse [Wed, 21 Feb 2018 21:42:05 +0000 (21:42 +0000)]
Cope with RTP as application/x-srtp by depayloading for ourselves
Farstream's FsRtpConference always installs the srtpenc/srtpdec elements
even when there is no encryption. In that case they just pass packets
through unmodified. Unfortunately the caps are negotiated wrong — it's
still marked as application/x-srtp. This means that rtpopusdepay won't
accept it, for example.
David Woodhouse [Tue, 20 Feb 2018 21:33:38 +0000 (21:33 +0000)]
Create RTP directly for incoming audio
It doesn't look like GStreamer/Farstream can cope with Opus with an RTP
clock rate other than 48000, but for incoming we can cope by just
multiplying by three.
David Woodhouse [Fri, 2 Feb 2018 20:59:13 +0000 (20:59 +0000)]
Switch to using RTP for audio
We get a jitterbuffer for free this way, which makes it a lot more usable.
It also means I can drop a lot of the timestamp mangling crap and just
transpose the protocol's sample counts into RTP headers... I think. The
idea is to drop the 'opusrtppay' part and feed RTP directly into the
appsrc, and the converse for the sink.
It does require farstream 0.2.8 *and* to remove libgstsrtp.so for now.
David Woodhouse [Mon, 11 Dec 2017 15:42:28 +0000 (15:42 +0000)]
Let fs-app-transmitter launch a pipeline by its description
Instead of hard-coded audiotestsrc/filesink, let the pipeline descriptions
get passed in in the FsCandidate's 'host' and 'username' fields, much as
the FsShmTransmitter does for the filenames.
This might not last, as we can only reference named elements in a pipeline
description if they were created in that same pipeline, AFAICT. So my plan
of pre-creating the appsrc and then referencing it from the description
probably isn't going to work.
Which means we might end up with a dedicated FsChimeTransmitter after all,
but hey, at least I have a better understanding of how some of this stuff
fits together...
David Woodhouse [Sat, 9 Dec 2017 09:01:06 +0000 (09:01 +0000)]
Start cleaning up audio support, handle mute
There are locking issues here. The appsink_new_sample() function can get
invoked from another thread, which means it can race with the timer.
I think if the g_source_remove() in do_send_rt_packet() runs while a
timed_send_rt_packet() is waiting for the rt_lock, it can deadlock.
The simple fix is probably to shift all the actual packet processing into
a separate thread with a simple packet-sending loop which does both audio
and muted frames.
Or maybe we don't want a thread wakeup to add to the outbound packet
latency, and we'll allow appsink_new_sample() to send directly for itself
with some simple locking scheme between it and the thread.
The thread can then handle DTLS and auth and everything else sequentially,
which lets me do DTLS "natively" with GnuTLS and saves me from lamenting
the absence of GDtlsClientConnection too much.
Guilherme Melo [Sat, 25 Nov 2017 16:22:06 +0000 (18:22 +0200)]
Fix attachment messages when the user sends it from other clients
On an IM conversation between "A" and "B", if "A" sent an attachment
from other client (e.g. Web), "A" would see the message on pidgin as
being sent by "B".
Also "A" would receive a notification from pidgin for its own
attachments.
David Woodhouse [Wed, 6 Dec 2017 16:37:41 +0000 (16:37 +0000)]
Call cleanup
Split out chime-call-transport.c so we can add DTLS more easily. Deal with
opening/closing the "Call Participants" window by adding a chat menu, which
will also be the basis for starting an audio call.
David Woodhouse [Tue, 5 Dec 2017 13:34:32 +0000 (13:34 +0000)]
Fix support for more than 100 contacts
Oh joy, there's a completely separate way to handle fetching of contacts
in stages. The 'NextToken' is in a special header instead of in the JSON
response, but ultimately it's very similar. Make fetch_contacts() work
like fetch_conversations() and fetch_rooms() do.
This fixes the disappearance of contacts, when you have more than 100.
Which was partly masked by the fact that we weren't deleting the buddies
from Pidgin.
David Woodhouse [Tue, 5 Dec 2017 13:32:10 +0000 (13:32 +0000)]
Fix buddy removal when they're no longer in the contacts list
We're supposed to remove purple buddies when they are no longer in the
contacts list, but we weren't removing them if they were visible to us
at all (by virtue of being in a conversation or a room).
David Woodhouse [Tue, 5 Dec 2017 13:30:04 +0000 (13:30 +0000)]
Explicitly close ChimeConnection in chime_purple_close()
Instead of allowing it to be implicitly closed by the final unref, close
it explicitly. This allows us to fix up the refcounting elsewhere, so
the objects associated with the connection can just hold a ref to it.
It was kind of OK for rooms/conversations but it's getting silly with
meetings adn calls.
Isaac Jurado [Thu, 16 Nov 2017 11:22:34 +0000 (12:22 +0100)]
Escape the "|" character in GWT requests.
Apart from the backslash "\", the pipe character "|" also needs to be escaped
for obvious reasons. Not escaping it breaks the format of the GWT requests.
Nicola Girardi [Wed, 25 Oct 2017 22:22:29 +0000 (23:22 +0100)]
Initial support for receiving attachments
- uses purple_util_fetch_url_len() rather than hijacking the soup session
- saves attachments in $HOME/.purple/chime-purple/$email/downloads
- displays attachment in the chat or conversation if content type is image
- notifies of downloaded file if attachment is not an image
- does *not* handle variants
David Woodhouse [Tue, 7 Nov 2017 04:55:43 +0000 (05:55 +0100)]
Remove 'wait-for-connection' property
Now we should have FsAppTransmitter "working", at least with the
audiotestsrc and filesink for now. Hooking it up to an actual
appsrc/appsink and to paulbohm's code remains to be done...
David Woodhouse [Fri, 3 Nov 2017 08:30:52 +0000 (08:30 +0000)]
Not working with filesink
The fsappbin1 bin never entes the PLAYING state, for some reason. Hopefully
I can get a coherent response to
https://lists.freedesktop.org/archives/farstream-devel/2017-November/000079.html
David Woodhouse [Wed, 18 Oct 2017 14:49:54 +0000 (15:49 +0100)]
Abort connection when server kicks us off
Oops, this snuck in with another commit. I'm willing to rewrite
history to fix that, even though I wouldn't do it to fix the stupid
"calender" typo that inexplicably slipped in last week.