By far my most cited paper in RL is the following. Rich Sutton is the first author and he understood the context and significance. I helped with some proofs. I am only recently appreciating the significance of this paper.

<p><a href = https://scholar.google.com/citations?view_op=view_citation&hl=en&user=nbpafUkAAAAJ&citation_for_view=nbpafUkAAAAJ:u-x6o8ySG0sC>
Policy Gradient Methods for Reinforcement Learning with Function Approximation</a> with Yishai Mansaur, Rich Sutton, and Satinder Singh, NIPS, 1999.</p>

Here is another RL paper from my time at ATT:

<p><a href = https://scholar.google.com/citations?view_op=view_citation&hl=en&user=nbpafUkAAAAJ&cstart=40&citation_for_view=nbpafUkAAAAJ:M3ejUd6NZC8C>
Approximate planning for factored POMDPs using belief state simplification</a> with Satinder Singh, UAI, 1996.</p>