I may not be understanding what you're asking, but...
HTML Code:
<ul>
<li><a href="#">That</a></li>
</ul>
Code:
ul a {
display: block;
padding: 5px;
margin: 0 0 5px 0;
background: url(/path/to/image.gif) no-repeat right center;
}
That should work. Just make sure your image is exactly a mirror of itself or it'll be off center. And make it a lot taller than it needs to be so your nav arrows can expand.