Bhagavad Gita: Chapter 2, Verse 34
अकीर्तिं चापि भूतानि
कथयिष्यन्ति तेऽव्ययाम् |
सम्भावितस्य चाकीर्ति
र्मरणादतिरिच्यते || 34||
akīrtiṁ chāpi bhūtāni
kathayiṣhyanti te ’vyayām
sambhāvitasya chākīrtir
maraṇād atirichyate
People will speak of you as a coward and a deserter. For a respectable person, infamy is worse than death.
For respectable people, social prestige is very important. The particular guṇas (modes of nature) of warriors make respect and honor especially important for them. For them, dishonor is worse than death. Shree Krishna reminds Arjun of this, so that if he is not inspired by superior knowledge, he may at least be inspired by inferior knowledge.
Many societies in the world observe the norm that a warrior who runs away from the battlefield out of cowardice is ostracized from society. That could be the extent of dishonor inflicted upon Arjun if he avoided his duty.