I think parallel is the best way to go, but doing it the way I want to is a bit tricky in pure TeX. After much thinking, I ended up writing a bash script to do it. It's not the most elegant solution (you can call it ugly if that makes you feel better), but it does the trick.
Here is my script:
#!/bin/bash
CHAPDIR="chapters"
SPLITDIR="splits"
# Split all files
for l in fr en; do
mkdir -p $SPLITDIR/$l
for c in $CHAPDIR/$l/*.tex; do
f=$(basename $c)
num=${f%%_*}
csplit --quiet --prefix $SPLITDIR/$l/${num}_ --suffix-format=%02d.tex --elide-empty-files $c /^$/+1 {*}
done
done
# Generate document
for c in $CHAPDIR/fr/*.tex; do
f=$(basename $c)
chnum=${f%%_*}
sfile="$SPLITDIR/${chnum}.tex"
# Make chapter title
#echo "\input{$SPLITDIR/en/${chnum}_00}" > $sfile
#sed -i 's@\\chapter@\\chapstyle\\chapheadstyle@' $SPLITDIR/fr/${chnum}_00.tex
#echo "\input{$SPLITDIR/fr/${chnum}_00}" >> $sfile
enchap=$(sed -e 's@\\chapter{\(.*\)}@\1@' $SPLITDIR/en/${chnum}_00.tex)
frchap=$(sed -e 's@\\chapter{\(.*\)}@\1@' $SPLITDIR/fr/${chnum}_00.tex)
echo "\\chapter{$enchap \\\\$frchap}" > $sfile
echo '\normalfont' >> $sfile
echo '\begin{Parallel}{2in}{2in}' >> $sfile
for pfr in $SPLITDIR/fr/${chnum}_*.tex; do
secname=$(basename $pfr .tex)
secnum=${secname##*_}
[[ "x$secnum" = "x00" ]] && continue
pen=${pfr/fr/en}
# Replace chapter with \Huge, section with \Large
sed -i 's@\\section\*@\\secstyle@' $pfr
# Remove lettrines
sed -i 's@\\chlettrine{\(.*\)}{\(.*\)}@\1\2@' $pfr
if [ ! -f "$pen" ]; then
echo "E: Missing file $pen"
exit 1
fi
echo '\ParallelPar' >> $sfile
echo "\ParallelLText{\selectlanguage{english}\input{$pen}}" >> $sfile
echo "\ParallelRText{\selectlanguage{french}\input{$pfr}}" >> $sfile
done
echo '\end{Parallel}' >> $sfile
done
I've moved my translated text to chapters/fr/ and added original chapters in chapters/en/. The script creates a splits/fr/ and splits/en/ directories (for the note, splits/ has got an 's' because I've got a rule called "split" in my Makefile), splits the chapters in chapters/* by paragraph (using csplit) and reassembles the chapters in splits/, using the parallel package.
The master document then contains the call to load the parallel package, as well as the \input calls:
% All chapters
\pagewiselinenumbers
\input{splits/01}
\input{splits/02}
\input{splits/03}
\input{splits/04}
Running ./split.sh followed by pdflatex masterdoc.tex generates the document with split chapters.
And here is how it renders:

paralleldocumentation and learned of a design decision that may affect whatever you do: its macro\ParallelParis intended to provide for coordinated side-by-side paragraphs. Thus, corresponding chunks may be different sizes but not fall too much out of sync via judicious use of this command. It seems to me that you will have to do something similar whether or not you useparallel, or else your side-by-side document will be a little confusing. Basically, your two files need to be written with parallelization in mind. – Ryan Reich Apr 25 '11 at 20:48\ParallelParinparallel's documentation. It also says that it's been suggested to make it automatic instead of explicit, which would be much easier for me since I do want paragraphs to correspond properly. Using\ParallelParexplicitly would make it quite hard for me though, without modifying the translated text. – raphink Apr 25 '11 at 21:07