We don’t want to watch you watching.
Sometimes while writing scenes using several players, an author needs to make it clear who’s talking. The most common way to do this is with attributions, those dialogue tags such as said or shouted. (I love said, which has a quiet dignity inside dialogue.) Beyond attributions, there are also places where we want to be shown a character doing something as a way of following who’s speaking.
Karen chopped the onions. “I think I can cut that man out of my life. Slice him away as neat as these Vidalias.”
That chopping is easy to visualize. Looking, on the other hand, is broad and fuzzy. There are many expressions. Having one character look to another and speak is just not as strong, unless you can charge that look with specifics. “She flashed her a knowing look” is stronger than “She looked at Ginger.”
You’re using that looking as sub-choreography. That’s a kind of a shadow stage direction to tilt the story’s camera in the direction of somebody else. Take as many of those out as you dare.
While you’re looking over your writing, you can also pay attention to how often the word look gets in the way of what readers see. Words like looked, watched, and saw are not an active element in any action that readers can observe. If you’ll take the looking out of your stories, and just tell me what’s in the frame, it’ll be more direct. I’m closer to the action that way, than being told who is observing it.
Using these looks can be a sign that your writing is reluctant to trust the reader. It is better, whenever possible, to let characters’ behavior and what they say convey the hidden meanings of their actions. This is one of the joys of reading and writing fiction. Readers get to supply subtext to the scene and the story, rather than absorbing deductive writing that erases all possibility of confusion. Nonfiction does that kind of fastidious cleanup. While you ensure that I don’t get anything wrong, you can’t help but leave your shadow in the frame of the scene that appears on the page.