![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
jmvalin
This is a follow-up on the first LPCNet demo. In this new demo, we turn LPCNet into a very low-bitrate neural speech codec (see submitted paper) that's actually usable on current hardware and even on phones. It's the first time a neural vocoder is able to run in real-time using just one CPU core on a phone (as opposed to a high-end GPU). The resulting bitrate — just 1.6 kb/s — is about 10 times less than what wideband codecs typically use. The quality is much better than existing very low bitrate vocoders and comparable to that of more traditional codecs using a higher bitrate.
Crash on raspi
Date: 2019-03-30 07:43 am (UTC)Thank you for the new codec. I have tested it on my desktop without avx2 and like the quality/space tradeoff. It's painfully slow decoding without avx2, but that is expected. I tried to cross-compile for raspberry pi 3 and get a crash in sincos. Do you know what I'm doing wrong? I'm using gcc-8.3.0 and I had to do a bit of work to get it to compile:
CC=arm-unknown-linux-gnueabi-gcc-8.3.0 CFLAGS='-O3 -ggdb -march=armv8-a -mfpu=neon' ../configure --target=arm-unknown-linux-gnueabi --host=x86_64-pc-linux-gnu
I'm on commit 343e35.
Re: Crash on raspi
Date: 2019-03-30 03:24 pm (UTC)About the code being slow without AVX2, if you don't have an AVX2-capable CPU, it's probably not very hard to convert the AVX2 code into SSE with very little performance loss. I haven't benchmarked, but I suspect even a Core 2 should be able to run LPCNet in real-time with SSE code.
Possible to parallelize?
Date: 2019-04-04 02:16 pm (UTC)Ist it possible to parallelize the neural-network (or even build it in a decent FPGA)?
Thanks!
Re: Possible to parallelize?
Date: 2019-04-04 04:25 pm (UTC)When it comes to FPGAs, I suspect LPCNet can be implemented quite efficiently, bot only if the weights of the sample rate network can fit on the chip and not have to come from external memory. That means having around 100k-200k internal storage.
Re: Possible to parallelize?
Date: 2019-11-22 12:42 am (UTC)If this would work it could mean that it would work on a R Pi in realtime.
Best regards
- Martin
Re: Possible to parallelize?
Date: 2020-06-08 05:14 pm (UTC)Opus
Date: 2019-04-10 11:18 pm (UTC)Re: Opus
Date: 2019-04-10 11:46 pm (UTC)Re: Opus
Date: 2019-04-11 05:13 am (UTC)Impressively practical
Date: 2020-02-19 05:00 pm (UTC)Can't get the same result
Date: 2020-02-25 04:14 am (UTC)Re: Can't get the same result
Date: 2020-02-25 05:22 am (UTC)Re: Can't get the same result
Date: 2020-02-25 08:02 am (UTC)PESQ.exe +16000 +wb ref.pcm ref_dec.pcm
PESQ.exe +16000 +wb ref.pcm lpcnq.pcm
I don't know if it is because of my compile, because a lot of warnings are generated, such as:
In file included from src/lpcnet_dec.c:38:0:
src/pitch.h:37:1: warning: C++ style comments are not allowed in ISO C90 [enabled by default]
//#include "modes.h"
^
src/pitch.h:37:1: warning: (this will be reported only once per input file) [enabled by default]
src/lpcnet_dec.c: In function 'decode_packet':
src/lpcnet_dec.c:96:3: warning: ISO C90 forbids mixed declarations and code [-Wpedantic]
unpacker bits;
^
src/lpcnet_dec.c:108:3: warning: C++ style comments are not allowed in ISO C90 [enabled by default]
//fprintf(stdout, "%d %d %d %d %d %d %d %d %d\n", c0_id, main_pitch, modulation, corr_id, vq_end[0], vq_end[1], vq_end[2], vq_mid, interp_id);
^
src/lpcnet_dec.c:108:3: warning: (this will be reported only once per input file) [enabled by default]
src/lpcnet_dec.c:135:3: warning: ISO C90 forbids mixed declarations and code [-Wpedantic]
float sign = 1;
^
CC src/lpcnet_enc.lo
In file included from src/lpcnet_enc.c:38:0:
src/pitch.h:37:1: warning: C++ style comments are not allowed in ISO C90 [enabled by default]
//#include "modes.h"
^
src/pitch.h:37:1: warning: (this will be reported only once per input file) [enabled by default]
src/lpcnet_enc.c:46:1: warning: C++ style comments are not allowed in ISO C90 [enabled by default]
//#define NB_FEATURES (2*NB_BANDS+3+LPC_ORDER)
PS:
The voice bandwidth of the "MELP 2.4kbps" downloaded from the website is 4kHz, but the MELP 2.4kbps program I downloaded from the network generates a voice bandwidth of 8kHz.
Re: Can't get the same result
Date: 2020-02-25 06:50 pm (UTC)Re: Can't get the same result
Date: 2020-02-26 01:47 am (UTC)Re: Can't get the same result
Date: 2020-02-26 08:10 am (UTC)Re: Can't get the same result
Date: 2020-02-26 08:47 am (UTC)Re: Can't get the same result
Date: 2020-02-25 08:40 am (UTC)Re: Can't get the same result
Date: 2020-02-25 11:00 am (UTC)the resudual sequence
Date: 2020-11-18 10:43 am (UTC)for (i=0;i
for (i=0;i<FRAME_SIZE;i++) {
int j;
float sum = aligned_in[i];
for (j=0;j<LPC_ORDER;j++)
sum += st->lpc[j]*st->pitch_mem[j];// st->lpc is -st->lpc?
RNN_MOVE(st->pitch_mem+1, st->pitch_mem, LPC_ORDER-1);
st->pitch_mem[0] = aligned_in[i];
st->exc_buf[PITCH_MAX_PERIOD+i] = sum + .7*st->pitch_filt;
st->pitch_filt = sum;
//printf("%f\n", st->exc_buf[PITCH_MAX_PERIOD+i]);
}
Opus and LPCNet
Date: 2021-04-08 08:15 pm (UTC)I noticed you moved from Mozilla to Amazon. Are you still planning to make this part of Opus? I think the need for this type op codec on devices with less capable procs is so valuable. I would love to hear how you see your work moving forward and maybe even what help and resources you would need to make this happen.
Cheers,
Mark Vletter