The Blues started with 10 new faces from their last Premier League line-up and struggled to find any rhythm in the first half as Barnsley refused to be cowed by their big-name opponents and created the better early chances.
New Chelsea manager Thomas Tuchel changed his side's shape at the start of the second half, bringing on full back Reece James to replace Marcos Alonso and centre half Antonio Rudiger for Andreas Christensen, and switching to four in defence.
"We did it well at the start of the second half and played constantly in the opponents half," Tuchel said after the match. "We scored the goal and I had the feeling that they increased the risk but we stepped down in the little decisions. Not enough courage and it was too deep for a long, long time.
"We clearly can play better, so yes I expected more but not in terms that I am frustrated."
The decisions paid off when James, fed by 19-year-old Billy Gilmour, raced into space and his cross found Abraham for an easy finish as Barnsley's defenders appealed in vain for an offside decision.
Video replay appeared to show Abraham marginally offside but the match did not have VAR as non-Premier League venues do not utilise it.
"Coming here is never easy. Give credit to them they are a good side. I knew chances would come eventually I just had to be in the right place at the right time," Abraham said.
Barnsley substitute Michael Sollbauer almost equalised immediately after coming on but his goal-bound flick was cleared off the line by Abraham with a full-stretch header in the 78th minute.
"We know that we can compete at that level. Chelsea could have had a penalty, the goal could have been offside, they are 50/50. Tonight we go to bed with a good feeling," Barnsley manager Valerien Ismael said.
The win was the fourth in five games for Chelsea since Tuchel replaced Frank Lampard last month, and it was the fourth clean sheet for the Blues under the German.
Last year's beaten FA Cup finalists will host Sheffield United in the next round, having beaten the Yorkshire side twice in the league this season including last week's 2-1 away win.