It was Ipswich who started the game on top and forced a decent save out of Liverpool goalkeeper and eventual man of the match Dave Roberts.
Liverpool soon found their form however and had chances of their own as Paul Anderson fired into the side netting, Jimmy Ryan blasted a free kick into the Ipswich wall and Adam Hammill tried his luck from 30 yards.
Ipswich continued to have chances of their own however, and Roberts was called into action again to make a couple of decent saves before half time.
Liverpool made a subsitution six minutes into the second half with Charlie Barnett coming on for Jimmy Ryan, and the change brought instant dividends when Barnett, with his first touch, found Anderson on the edge of the Ipswich box. The former Hull winger buried his shot into the bottom corner of the net to give the Reds the lead.
However, the Reds held the lead for a mere ten minutes before Ipswich were level. Stuart Ainsley hammered a left foot volley past Roberts, who could do nothing about the equaliser.
Liverpool continued to press and Hammill again came close to scoring when he saw a shot tipped around the post for a corner.
Liverpool's continued pressure paid off on 72 minutes when Adam Flynn picked up Godwin Antwi's through ball in the Ipswich area and calmly slotted home what would be the eventual winner.
Ipswich didn't give the game up and continued to push for an equaliser. They had the ball in the net ten minutes from the end but it was disallowed for offside to the relief of the crowd.
Ipswich had one final chance in stoppage time but again Roberts was equal to it, saving the ball with his left foot to prevent extra time and ensure the Reds progressed to round five where they will meet Burnley.