The violence erupted overnight in the El Abra prison outside the city of Cochabamba and police lost control of the jail for several hours as inmates attacked each other after an alcohol-fueled party.
"So far we have counted four dead and 11 wounded. The dead have now been removed and the wounded taken to hospital," said Andres Cuevas, a regional government official.
Rivalry between Bolivian and foreign prison gangs was to blame, according to the regional police commander, Alberto Suarez.
"Because of these power struggles, fighting has broken out in which unfortunately people have been wounded by knives and firearms," Suarez told Radio Fides.
He did not name the nationalities of the foreign inmates. Bolivian media reports said they were Colombians jailed for drug trafficking.
The four dead were all Bolivian and included a prison gang boss named Ariel Tancara, whose throat was reportedly slit.
Bolivian TV channels showed images of his sister outside the prison screaming: "I want my brother's body! Give me my brother!"
Witnesses said the inmates had organized a party Sunday night that included alcohol and live music.
Inmates' family members condemned prison officials for allowing alcohol into the prison, accusing them of corruption.
It is the worst prison violence in Bolivia since August last year, when 35 inmates were killed in riots at Palmasola prison in the eastern city of Santa Cruz.
Bolivia's prisons are the most crowded in Latin America after El Salvador's, according to the Organization of American States.
Its jails have the capacity to hold 3,738 prisoners, but currently house 13,840, OAS reports.
Stay updated on the go with The Times of India's mobile apps. Click here to download it for your device.