Expanding on the answer of @jrg and @Richard, I think we can safely assume that we should consider the question as referring to times beginning from 1980s, since that's when UNIX systems actually appeared on the mainstream user/corporate market. Obviously neither *n*x OS was an option earlier than this rough date due to movie/comix readers/watchers (not to mention creative staff's) perception of the whole computer science; we got mainly blinkenlichten in movies from pre-1980 era due to little recognition of actual computer operating systems' looks.

To sum up:

a) Since we can assume Batcomputer was a mainframe during 1980-1990 era, it's highly probable it was shipped with some custom UNIX clone, however it is not unheard of entirely machine-specific systems, especially with high-end machines. He certainly hasn't used Linux during that time, because Linux hasn't been created by Torvalds yet.

b) Beginning with ca. 1995, Batman used T932, a most powerful variant of T90 supercomputer, running UNICOS, which is in turn based on UNIX ("CRAY T90 systems run Cray Research's industry-leading UNICOS operating system based on UNIX system V"); we can safely consider he ended doing so somewhere about 2005, when it was obsoleted by Cray X1; thus he most probably wasn't using Linux for this time (even though Linux already existed).

c) Batman's Wayne Enterprise Mainframe (which is, most probably, used as the Batcomputer, since most of Batstuff is actually disguised as actual corporate property one or another) uses *n*x OS currently; see

Wayne Enterprise Mainframe:~ en$ nmap -v 192.168.23.10

with

username:path$

being the common command prompt of e.g. bash shell.

Since custom *n*x OSes are ATM the usual choice for supercomputers nowadays, mainly due to maintainability and security provided, it's quite obvious Batman would choose one for his Batcomputer. Hard to say if it's Linux or Unix/BSD clone due to lack of data - but saying it's a Unix-like OS is quite a good guess IMO.

(as as side note, I'd also assume it's a custom-tailored, highly personalized distribution - mostly because that's the way a power admin would go with his personal mainframe)