The hosts went ahead when Smith headed in Youssouf M'Changama's cross.
Liverpool skipper Luis Suarez was allowed to run at the Oldham defence as he squeezed in an equaliser.
But Smith took advantage of keeper Brad Jones's error and Reece Wabara scored a delightful back-post header as Oldham restored their advantage although Joe Allen's volley ensured a tense finish.
Steven Gerrard, who was brought on from the substitutes bench with 35 minutes remaining in attempt to turn the game around, hit the bar as nerves frayed at a wet and windy Boundary Park.
Yet Paul Dickov's side held on for a superb win over the seven-times winners on a weekend when Football League sides have caused upsets on all three days of the fourth round.
Oldham have been beset by financial problems in recent months, meaning manager Dickov has had to shed much of his coaching staff but his players produced a display which made a mockery of their lowly position in the third tier.
It also completed a fantastic response to their 5-1 defeat by the five-times European champions in the FA Cup third round last season.
Despite Gerrard's omission, Liverpool were not under-strength with Suarez, Daniel Sturridge, Fabio Borini and Raheem Sterling all starting in attack.
However, they were overshadowed in the opening exchanges by superb combination play between Smith and his strike partner Robbie Simpson, who bullied Liverpool centre-backs Sebastian Coates and Martin Skrtel.
They made their mark, sometimes literally, and 23-year-old Smith showed his intentions as early as the third minute with a well-taken goal.
Lee Croft mis-cued a shot from 20 yards but it found its way out to the left where M'Changama was given time to cross and Smith beat Coates to head in despite Skrtel's effort to clear off the line.
The tie continued its ferocious start when Raheem Sterling scythed down M'Changama and Simpson followed in on Reds keeper Jones after he spilled a shot.
But as much as Oldham were on the front foot, they were also slow to retreat and they gave Suarez plenty of space to equalise when he ran from his own half and squeezed the ball in after playing an inadvertent one-two with Oldham defender Cliff Byrne.
Sterling should have given Liverpool the lead before the half-hour mark when he breezed into the box but the 18-year-old shot straight at Latics keeper Dean Bouzanis.
Jose Baxter was showing some neat touches in midfield for the hosts but it was against the run of play when Oldham went ahead once more in first-half stoppage time, although it looked as though Sturridge may have been fouled in the build-up.
There looked no danger when Wabara weakly crossed the ball from the right, but Australian keeper Jones fumbled and Croft squared the ball to Smith who tapped in from close range.
Borini could have brought the visitors level again straight from the second-half kick-off when he scooped the ball over the bar and it became a crucial miss when Wabara excellently headed in Winchester's deep cross from the left.
That was enough for Liverpool manager Brendan Rodgers to bring on Gerrard, and the Liverpool skipper made an instant impact as his team created several good chances.
Superb blocks from Byrne and Baxter denied the England midfielder and although Simpson shot wide as Coates continued to struggle with his presence, the visitors got a goal back from a corner when Allen volleyed in from the edge of the box via a deflection with 12 minutes remaining.
It looked as though it would prompt a late Liverpool surge, but as Oldham clung on they recorded yet another shock on a magical FA Cup weekend.