Steve and Dustin were talking about new processes and using some new and fancy display options. After that, I helped Steve with some JavaScript calculations and some small page changes.

Wayne popped in and he gave a report about some of the server issues that happened yesterday. It sounds like a company was trying to pull multiple queries that had 2 million records per query. Basically, a log jam in the database. As we were talking about things, we were talking about moving from a purely transactional database, to a more aggregate type environment (aggregate meaning sums and totals vs individual transactions). Tons of data is awesome, but we need to limit the transactions and how many we are trying to chew up per bite. Some of the reports need to be re-written to be more efficient. Nobody wants to read and look at 2 million lines... all they really want is the totals, the counts, and the sums from that data. We may need to filter the data and better display it in a drill-down type basis.

Wayne was talking about DAO's (database access objects) and getting the queries in a more standard location and where to access those assets.

We are making progress and making things more stable. Sometimes when our users see an error, they instantly go back to what things were (older past errors) and then they fear the worst. That is just human nature. These are some of the growing pains. As a funny side note... Wayne was saying, other companies that seem light and fancy, they will have the same issues as they grow, their day will come when they are up against the wall and have to move from transactional data to a more aggregate type model.

Wayne was also talking about dynamic services vs just static boxes (dedicated servers). Basically a way to throttle resources vs setting up a static environment with specific resources. We were also talking about roll over protection and other mirror type values. We lightly were talking about how we may be out growing our current hosting company. We may start looking into AWS (amazon web services), Google, and/or other bigger players. Currently we are using Newtek and they have been awesome for years and years.

There was also some talks and discussion about a new pricing model based on usage, storage, bandwidth, requests, etc. We would love to get there but it still seems to be out there a bit. We were also talking about stats and then also projecting those stats out to our users. Some of this may require us to setup things per instance or per client. Random side note, I wonder if we were to go back to the drawing board and start with the end goal in mind. Would that change things? How, why, and how deep? Do we want that?

Maybe help our clients run based on sizes and needs. We grew from a small shared environment into a semi dedicated model and then into a fully dedicated model. We now have multiple dedicated servers. Wayne was saying something about Docker (spelling) containers and how to group things and somewhat create a virtual playground or virtual cluster per corporation or per company. Once we get them into the container of sorts, we could then expand that contain as needed. SaaS (software as a service) and how to expand and grow things.

Funny quote from Steve - "We need to get Wayne on a faster horse." Meaning, we may need to help Wayne get on an environment where he is not so limited (existing dedicated servers, structure, and physical boxes). Maybe we need to move more to the virtual playground.

Steve had some ideas about using more of the invoice homepage type mentality. Basically, we had an older report that pulled in the last 30 days worth of transactions. We totally changed that page to pull quick counts, graphs, charts, and sums. We then only show the last 15 records (really easy on the system). They users can flip/flop between daily, weekly, monthly, quarterly, yearly reports at the click of a button. All of the values are queried and pulled using a technology called AJAX (go get just the data and return without refreshing the whole page). Then, if they want more details (actual transactions), we have a number of other reports where they can filter the data to smaller and smaller levels. Steve would like us to do more and more of that to help eliminate the full detailed transaction searches. Great idea.

Towards the end of the session, I spent some time and built in a pre-check query for the advanced invoice line items search. If it has more than 100,000 records trying to return, it shows an error and requires the user to filter the details down a little bit. Apparently, this was the query that brought down data 8 yesterday for a bit. The company that was running the query tool was searching for over 2 million records in a single report. They were running back to back queries of the same size. That big of a bite created a data log jam in the database (table locking for joins then processing and doing calculations on 2 million plus records per query). That eventually caused us to reset the whole server. As a side note, that same query tool joins about 5 tables in order to get the results. If you do that for a couple thousand records, no problem... if you do it for millions and millions, at a time, it is a problem.