A memory bank was any device intended for holding information over an extended period. While the term implied only computer-based devices, it technically included hard copies, cave drawings, and others. Physical sizes of memory banks ranged from a hand-held data module measuring as little as a half-inch, to massive constructs the size of buildings, with data capacity ranging accordingly.
Disaster recorders of Earth starships were equipped with both microtapes and memory banks. The banks acted as a backup if the tapes were destroyed. In 2265, Lieutenant Commander Spock recovered data about the fate of the SS Valiant from these banks after it had been determined that the tapes were burned out. (TOS: "Where No Man Has Gone Before")
The memory banks of the alien probe Tan Ru were transferred and merged with those remaining in the Earth probe Nomad when the two spacecraft met in deep space. This created a probe with the unintended directive to "seek out and sterilize new life," with Nomad as the body and predominant voice during their co-existence. (TOS: "The Changeling")