The Markit/Nikkei services Purchasing Managers' Index (PMI) for Japan decreased to 49.3 in April from 50.0 in March.
A reading below 50 indicates contraction of activity, while a reading above 50 indicates expansion.
The decrease was driven by a reduction in clients.
"Latest survey data indicated a slight deterioration in output at Japanese service providers. Activity decreased for the first time in over a year, while new orders were broadly stable during the month," economist at Markit, Amy Brownbill, said.