This paper aims to provide insight into the interplay of algorithmic settings and the possibility to customize recommendation into perceived agency. In news recommender systems users can exercise implicit control through the feedback loop and explicit control through adjusting settings. So far there is still little research into if and how users exercise active control. Using a novel experimental design (N= 248) that allows users to engage with a news recommendation system, we find providing the functionality of explicit control of a news recommender system to users leads to higher levels of perceived control but not to higher levels of satisfaction. Additionally, we also see that the control settings were only sparsely used. The findings are discussed in the light of recent work on agency in the datafied society.