If you're a serious baseball nerd and a serious local history nerd, you probably know that Olympia was a member of the national minor league system from 1903-1906. Sadly, you're also misinformed, as the Southwest Washington League didn't make it into 1906, but current records (incorrectly) indicate otherwise.
But, most important to me isn't really how the team played, but where exactly they played.
The above image shows the best guess of where at least the grandstands for the baseball field were. The map (from a great history on the Thurston County Fair) is a failed proposal for expanded county fair grounds on the site of Carlyon Park, where the baseball field was housed. The black triangle in the middle of the image indicates the grandstands of Athletic Park.
This article from the 1903 Morning Olympian points to a stop on the trolley line between Olympia and Tumwater run by the Olympia Light and Power Company. This piece in 1920 chronicles the end of life of "Athletic Park" right before it was torn down for the current residential neighborhood and replaced by what would become Stevens Field.
After the Olympia Senators (or Maroons, I'm not really sure) folded after 1905, Athletic Park played home to several semi-pro town team, industrial league teams and local school teams. The image below from the 1920 Olympia High School Annual, towards the end of Athletic Park.
Throughout 1903-1920, the grandstands of Athletic Park are almost totally absent from the pages of the Olympus (except for here), but these images show clearly the outfield wall and bleachers added to the park to round it out. The best image of the looming grandstands can be found here.
By the way, I was already working towards this conclusion a week or so ago, but a great discussion over at the Olympia Historical Society's Facebook page pushed me over the top.