The ReplayMod should, during recording, not influence performance at all. If I'm not mistaken, literally the only thing which could impact performance during recording under normal operation (given you have more than one CPU core) is the recording indicator in the top left and that's practically nothing.
Please post the contents of the fml-client-latest.log file inside the logs folder inside your .minecraft folder to some pastebin site and link to the post here.
Note that this log will only contain information on the last time you started Minecraft, not any previous runs.