MythTV Commercial Cut Script

I have been using MythTV to record OTA Broadcast television since March 2012 on my FreeBSD server. I recently updated to version 0.26 and found that the mythtranscode utility no longer functioned. From online discussions it is apparent I am not the only person affected. Anyways, I decided to experiment with my own commercial cut and transcode script to run as a user job. I previously had been using the mythfrontend edit mode to mark commercials, then mythtranscode to cut them and finally HandBrakeCLI to transcode to an x264 MKV file. When mythtranscode stopped working I experimented with ffmpeg and found that I could cut the video clips I wanted, transcode them and the combine them with mkvmerge. So that is what my bash script does.  I will break the script up to help explain the sections. Here is the scripts beginning:

#!/usr/local/bin/bash

#My attempt to cut commercials from myth recording and save as mkv
#Run as MythTV User Job with %FILE% as argument
#Argument 1: filename

#change variable below to your needs
logdir="/var/log/mythtv"
mythrecordingsdir="/store/mythtv"
outdir="/store/mythtv/video" 
scriptstarttime=$(date +%F-%H%M%S)
logfile="$logdir/$scriptstarttime-COMCUT.log"

#cleanup from last job
rm $outdir/tmp*.mpg*

echo "ComCut & Transcode job $1 starting at $scriptstarttime" >> "$logfile"

First Task: I need to get the cut points from the ‘mythconverg’ database table ‘recordedmarkup’. The CHANID and STARTTIME can be determined from the file name argument. Then using mysql command retrieve the query results.

#variables
USER="username"
PASS="password"
DB="mythconverg"
CHANID=$(echo $1 | cut -c 1-4)
STARTTIME=$(echo $1 | cut -c 6-19)

#Retrieve recording info from mysql
W=$(mysql -u$USER -p$PASS $DB --disable-column-names -e "SELECT data FROM recordedmarkup WHERE chanid=$CHANID AND starttime=$STARTTIME AND type=30;")
L=$(mysql -u$USER -p$PASS $DB --disable-column-names -e "SELECT data FROM recordedmarkup WHERE chanid=$CHANID AND starttime=$STARTTIME AND type=31;")
FPS=$(mysql -u$USER -p$PASS $DB --disable-column-names -e "SELECT data FROM recordedmarkup WHERE chanid=$CHANID AND starttime=$STARTTIME AND type=32;")
QRY=$(mysql -u$USER -p$PASS $DB --disable-column-names -e "SELECT type,mark FROM recordedmarkup WHERE chanid=$CHANID AND starttime=$STARTTIME AND (type=0 OR type=1) ORDER BY mark;")

Update: I think I need to add some more information about the QRY variable. It contains the type of cut and frame numbers for each cut. A type value of zero is  MARK_CUT_END and of one is MARK_CUT_START. (http://www.mythtv.org/wiki/Recordedmarkup_table) The columns are tab delimited. Below QRY is converted to an array where the first element is the type and the next is the frame number. See comments below.

Second Task: I need to convert the frame number of each cut to seconds for ffmpeg. This can be calculated by the equation FRAMENUMBER / FPS  = SECONDS.   I used the ffmpeg fast and accurate method to speed up the process.

#### ffmpeg Cut Src Files into clips ###
#convert QRY to array
array=( $( for i in $QRY ; do echo $i ; done ) )
SS=0.0 #SLOW SEEK
SKIP=0 #FAST SEEK 
cnt=${#array[@]} #NUMBER OF ARRAY ELEMENTS
for (( i=0 ; i<cnt ; i++ ))
do
 if [ ${array[$i]} == 0 ] #MARK_CUT_END
 then (( i++ )) #increment i to get the end frame number
  SS=$(echo "${array[$i]} / $FPS * 1000" | bc -l) #bc-l should return scale of 20
  SS=${SS:0:(${#SS}-17)} #get substring of SS with only 3 decimal places for milliseconds
 else (( i++ )) #MARK_CUT_START - increment i to get the start frame number
  #since start, calc the duration
  T=$(echo "(${array[$i]} / $FPS * 1000) - $SS" | bc -l)
  T=${T:0:(${#T}-17)} 
  #if SLOW SEEK is greater then 30 seconds the use FAST SEEK
  if [ $(echo "scale=0;$SS/1" | bc -l) -gt 30 ]
  then
    SKIP=$(echo "$SS-30" | bc -l)
    SS=30
  fi
  ffmpeg -ss $SKIP -i $1 -ss $SS -t $T -vcodec copy -acodec copy $outdir/tmp$i.mpg
 fi
done

Third Task: Loop through all the temp mpg files and convert to x264 MKV files using HandBrakeCLI.

#### HandBrakeCLI files ####
for file in $(find $outdir -name tmp*.mpg); do
echo "File Found - $file" >> "$logfile"
HandBrakeCLI -i $file -o $file.mkv -f mkv -e x264 --x264-preset superfast --x264-profile high --x264-tune film -q 30 -E lame --ac 2 --ab 128 --audio-fallback ffac3 --crop 0:0:0:0 -w $W -l $L --decomb
echo "HandBrakeCLI exit code:$?" >> "$logfile"
done

Fourth Task: Build the mkvmerge command requires all the file names and the append-to info so it can exit without warnings and mythtv sees the job completed. First loop through all files then execute mkvmerge.

#### MKVMERGE FILES ######
MERGE=" "
PLUS=""
APPEND="--append-to "
i=1
for file in $(find $outdir -name tmp*.mpg.mkv); do
echo "File Found - $file"
MERGE=$(echo "$MERGE $PLUS$file")
if [ $PLUS == "+" ]
then
APPEND=$(echo "$APPEND$i:0:$[i-1]:0,$i:1:$[i-1]:1,")
(( i++ ))
fi
PLUS="+"
done
APPEND=${APPEND:0:(${#APPEND}-1)}
echo "$MERGE $APPEND" >> "$logfile"
mkvmerge -o $outdir/$1.mkv $MERGE $APPEND
echo "mkvmerge exit code:$? " >> "$logfile"
echo "Job Finished! $(date +%F-%H%M%S)"  >> "$logfile"

I placed my script in the same directory as my recordings and gave it execute permissions. From mythtv-setup’s general section you can add a user job name and command. Obviously this script is dependent on ffmpeg, HandBrake, and mkvtoolnix. From mythfrontend I add my commercial cut and then run this script from the job menu. This is just an experiment so use cautiously.

6 Comments

  1. MGilbert

    I had the same issue with MythTV 0.26. I finally got mythtranscode to work by installing some older libraries, but I’m looking for an alternate solution. This solutions looks pretty slick, but I keep getting an error on:

    T=${T:0:(${#T}-17)} or SS=${SS:0:(${#SS}-17)}

    Typical values for T or S, in my case, prior to this line are 0, 5086, 31127, 53756, 82188, 104464, etc. Because of this, the above fails with ‘substring expression < 0’, since it can’t trim 17 characters off of the string.

    Did you experience this same behavior?

    Regardless, thanks for the script and the hard work.

  2. Marty

    This is great!

    Have you had any issues with:

    T=${T:0:(${#T}-17)} or

    SS=${SS:0:(${#SS}-17)} ?

    I get an ” substring expression < 0″ error. Have you experienced this?

     

    Thanks again!

    • Reuben Crane

      I have not had any errors from these lines. Sorry.

      I am running FreeBSD 8.3-RELEASE-p3 and bash version 4.2.37

      Note that the “bc -l” uses the default scale of 20 digits after the decimal. ffmpeg expects only 3 digits for the milliseconds. That is why I extract the substring without the last 17 digits. How many decimal places are you getting?

      • Reuben Crane

        Try this from shell and see if you get the scale of 20. In my script I tried using scale=3 but had issues getting that to work and I have forgot why I did this the way I did.

        $ echo “4/3” | bc -l
        1.33333333333333333333
        $ echo “4/2” | bc -l
        2.00000000000000000000

  3. Marty

    Sorry for the double post. Something weird was happening with my browser or the comments feature.

    I get the same numbers as you do using the bc example above. The problem in the script occurs when the array[$i] is 0. The calculation is (0 / $FPS * 1000) – $SS and bc -l returns 0.

    I know little about how MythTV stores the data in the recordedmarkup table. What I’ve done at this point, is wrap the offending line in a ‘if [ $T != 0]’ statement, to ensure it doesn’t fail. I’ll try to work on it more today and post my findings.

    Thanks again for the help!

     

     

     

    • Reuben Crane

      Your right ‘bc -l’ returns 0 when the result is zero. Hmmm, I think this has to do with your ‘recordmarkup’ start and stop times. Add a line after the QRY gets set by mysql like:

      echo “$QRY” >> “$logfile”

      Check out the logfile for the cuts. I am curious what you get.

Leave a Reply

Your email address will not be published. Required fields are marked *