It sounds like you have plans for another jam, and thatâs good. One thing you should definitely do if you want users to peer-judge next time is have someone create a package with all of the submitted games so that we can just drop that onto the device and play them.
I understand the challenges with organizing a jam. And not knowing how large your staff is, etc., Iâm wondering if you maybe only have yourself to assist with managing these kinds of things. However I am seeing issues that I noticed you did not highlight. As a participant, I appreciate you inviting us to give feedback. So I want to make sure these issues are clearly voiced:
-
The jam rules were clearly stated, and included specific criteria for judging the games. Thatâs great!
-
Unfortunately, the default voting categories on itch.io do NOT account for those criteria. And so, having the itch.io default criteria used for the âcompositeâ voting results leads to scores that are not representative of the entries or their quality. For example, many games submitted were arcade or puzzle games. And those accordingly tended to have little to no story. So a voter would probably say âhuh, they have no story? well itâs not a bad game, but I guess in that category theyâre 1/5 right?â for the story section. Which ends up heavily skewing the âoverallâ results.
-
Related, the criteria that were stressed in the rules were not part of the voting results, and had NO impact on the itch.io voting. For example, how the game plays on hardware, or whether it was even optimized at all for the GameShell was NOT a category that was voted on. âŚAnd that was the whole point of the jam 
-
I noticed that you mentioned that â1 or 2 judges ratings could not have changed the result significantlyâ â Unfortunately, that is not correct. Entries that did not receive many votes would have DEFINITELY benefited from judge ratings.
-
There was no consistency in voting â the rules gave the impression that there would be a panel of judges rating each game. This implies that each game would have been rated by the same set of 5 or however many people. This is a wonderful idea, because itâs the most fair. Instead, we had up to 5 or 6 (often less) people voting on any given game. However, it wasnât the same 5 or 6 users who voted on any given game, and there is therefore no consistent scale for the results.
If for example every submitter had voted on every game, this would be much easier to swallow. But instead, itâs pretty much certain that any set of folks that voted on a game were not the same 5 or less that voted on another.
So because guidelines were laid out, but not followed, it makes it feel much less like âvotingâ and more of just a semi-random raffle.
Anyway, the reason Iâm saying all this is that I do enjoy the Game Shell. And I think that at least technically the forum is great. It uses the same setup as the NextThingCo forums, and the forums were definitely a highlight of that company and of being a Pocket Chip user.
lol I really, really want people to make fun GameShell-compatible games. And I think itâs great that you want to reward them for doing so.
Please understand that doing things like outlining guidelines for a contest and then NOT using those guidelines, and offering no contact explaining this during the whole process is a horrible way to handle things.
If for example you had decided to let everyone know this was the plan DURING the voting period, then more people would have voted on games and we would have had a more accurate set of ratings to go by.
Or if you had just said âhey, we canât find judges⌠weâre going to delay the judging for another weekâ â people would have been fine with that.
Instead, changing the rules for the contest after the fact just cause your users and supporters to think that you really do not care about them or their time. That you do not value their work, and that you have no interest in sticking to your word. Itâs just very rude. And it definitely does not inspire confidence.
For example if a successor to the GameShell was announced right now, who would really be interested in picking it up day 1? Instead, based on the handling here, I wouldnât be surprised if a âGameShell 2â got released, and after the orders were fulfilled it starting coming out that there would be announcements of âohhh we didnât use the CPU we said we were going to⌠and it has less RAM⌠and no d-padâŚâ or âoops, the screen isnât the resolution we said; itâs a little smaller but so what?â or things like that.
Believe me, I would MUCH rather have spent this wall of text doing a tutorial showing people on here how to make neat games in pico-8, or writing a tutorial on how to setup different emulators on the GameShell. And itâs great that youâre saying users would potentially be rewarded for those efforts. But I really have to doubt that the statement you made about âawarding CPI regularly [to active members]â will happen when just sticking to your own jam guidelines proved to be such a challenge. Not trying to be disrespectful, just honest.
