We consider prefiltering for a single–carrier transmission over frequency–selective channels, where reduced–state trellis–based equalization is employed at the receiver, such as delayed decision–feedback sequence estimation (DDFSE) or reduced–state sequence estimation (RSSE). While previously proposed prefiltering schemes are based on the optimum filters
of decision–feedback equalization (DFE), the prefiltering scheme introduced in this paper is designed according to a signal–to–interference–plus–noise ratio (SINR), whose definition takes into account explicitely the subsequent trellis–based equalizer and its complexity. In addition to the prefilter, a finite–length target impulse response for DDFSE/RSSE and an infinite–length feedback filter for state–dependent decision feedback in DDFSE/RSSE, respectively, is optimized. The developed solutions lend themselves to an interpretation of the tasks of the optimum filters. The presented numerical results show that noticeable gains
can be achieved compared to state–of–the–art prefilters.