Thanks Tarun! What you write and our conversations made me consider the following mutually-incompatible but complementary issues at play in writing, both for fiction and non-fiction.
1. Creative Art for Passion vs. Business for Money
Many creative types who produce great work tend not to care much about business and money, while those who care about business and money tend not to dive into literary and philosophical themes for deeper or richer exploration. Thus the creative types produce amazing work that never gets noticed and thus they can’t pursue their passion that requires business thinking (marketing, distribution, monetization), while the business types write boring and inconsequential stuff (high volume or low quality for monetization).
2. Writing for Yourself vs. Writing for Audience
I agree with you that writing non-fiction is writing more for yourself since it’s a means of simply stating or convincing others of your beliefs, while your experience provides some data points that fiction is more writing for the audience since it’s more for emotional impact, and it’s important to keep the readership hooked and interested, since after all they’re paying you in terms of time, attention, and even money.
3. Non-Fiction vs. Fiction
The same point can be made both in non-fiction as a simple straightforward declarative statements, as it can in fiction with an imagined universe involving characters, plot, and setting. I’ve always written non-fiction up to now, but am finding that fiction has some notable advantages in that you can (1) construct arguments and present it in a way that flies under the usual defenses, and (2) you are less open to ideological heat for holding a political view.
4. Creative Generation From Nothing vs. Analytical Refinement of Something
Writing on a blank page to express new ideas in an uninhibited way is the exact opposite of analyzing an existing draft for correct grammar, readability, and even word count, and to edit it for a finished product. It seems “Writer’s Block” is tying to do both at once, when what’s required is mindfully focusing on one, and explicitly alternating from one to the other.