Hello all. Wanted to take some time to talk about the A.I. related issues in the fed this week.
For those who didn’t see, we posted an announcement in the Discord on Wednesday stating that we had found that A.I. tools have been used in promos and were offering those who had used it a chance to come forward and discuss this with the moderators. We would then think about the next steps based on these discussions. We had no intention of publicly naming those that were flagged and still don’t.
In terms of how promos were flagged, we checked using a tool used by one of the moderator's employers, which checks a passage of text (in this case, a whole promo) and returns a percentage score for how much the passage has been written by A.I. The vast majority of promos for the last couple of shows returned low scores (mostly 1% or lower) indicative of original work. Promos from three handlers were flagged as having minor or significant uses of A.I.
Of course, the tool's veracity is something we had little knowledge of (except it being a paid service used by a company, rather than a free online tool), and so we asked those who had used A.I. to come forward. We wanted to know if any others had used A.I. in substantial ways that hadn't been flagged to gauge the system’s veracity. The three handlers that were flagged by the tool all accepted to having used A.I. to varying extents, and the scores are correlated to the extent to which A.I. was used. Talking to those handlers helped ascertain how the use of A.I. looked and what level of input was required to generate the A.I. elements. No other handlers came forward other than those flagged by the tool (other than those who didn't know what we meant by using A.I. tools).
It seems like rephrasing and restructuring tools on Grammarly (or other A.I. systems), as well as automatic synonym replacement tools like Quillbox, were mostly responsible for the high scores. I didn't really know what these things were until earlier this week and I imagine a lot of you don't either. Just as an example of what a rephrasing tool can do, Tommy ran some of his Winter Wasteland promo through the one on Grammarly.
Here is the original, scoring less than 1% on the A.I. detection tool:
Rocco was startled out of his slumber around 4:30 on Saturday morning. Tommy was angrily shoving the Christmas tree that Randi had put up in the living room back into its box. For the first time that he could remember, he was actually looking forward to the holidays, but the nightmares and Keres’ looming presence in the apartment had ruined all of that. He had allowed himself to get excited about the idea of waking up on Christmas morning and watching Randi and Walker open gifts. He didn’t even care that Walker was too young to fully understand what was going on. He finally had the family he had never had before, but Keres had ruined it all.
It had been more than a week since Tommy had seen Randi and Walker. The minutes felt like hours, the hours like days, and the days like weeks. The man who had once enjoyed life as a bachelor suddenly missed the family that he had started. Of course, he wasn’t truly alone. Rocco had set up homestead in Tommy’s apartment the morning that Randi took Walker and fled to her aunt’s home in Tulsa.
And the reworked version, scoring 97% on the A.I. detection tool:
Rocco jolted awake around 4:30 on Saturday morning, startled by the commotion. Tommy was forcefully packing away the Christmas tree Randi had lovingly decorated in the living room. Despite his anticipation for the holidays, his excitement had been overshadowed by nightmares and the ominous presence of Keres in their apartment. He had been eagerly anticipating spending Christmas morning with Randi and Walker, relishing in the joy of giving gifts, even if Walker was too young to fully grasp the occasion. For the first time, he felt the warmth of having a true family, only to have it shattered by Keres' interference.
It had been over a week since Tommy last saw Randi and Walker. Each passing moment felt like an eternity, amplifying his longing for the family he had begun to cherish. Despite the sudden absence of his newfound family, he wasn't entirely alone. Rocco had taken up residence in Tommy's apartment since the day Randi left for her aunt's home in Tulsa, seeking refuge from the turmoil caused by Keres.
You can argue about which version is better but obviously the original is already very good. I imagine the results would be more marked when running something less refined through the same tool, and other tools from other A.I.s may be more or less powerful. Regardless, the difference is clearly noticeable to the point where I don't think the reworked version can be truly considered the writer's own work. Although there is no specific rule in our rules thread about using A.I. in this way, we feel there is an obvious expectation that work put out in a competitive setting is your own.
Although we do not intend to publicly name those handlers concerned, we will reach out to handlers who have lost to a promo that used A.I. in a significant way, such as the use of rephrasing/restructuring tools. This is not a large number of people. We will discuss with them if there are any ramifications for their character. I also just wanted to make clear that no mod character is involved in this as an opponent (successful or otherwise) of a promo that was flagged as having A.I. I should also point out that this is completely unrelated to Cap’s decision to step away from the fed. Cap’s promos were not flagged as having used A.I. in any way and Cap is welcome back to the fed if he should ever choose to roleplay here again.
Going forward, we would like to establish what is considered okay and what isn’t:
You can reach out to a mod if you have any questions or thoughts or concerns you’d rather share privately.
Thanks
Mods
For those who didn’t see, we posted an announcement in the Discord on Wednesday stating that we had found that A.I. tools have been used in promos and were offering those who had used it a chance to come forward and discuss this with the moderators. We would then think about the next steps based on these discussions. We had no intention of publicly naming those that were flagged and still don’t.
In terms of how promos were flagged, we checked using a tool used by one of the moderator's employers, which checks a passage of text (in this case, a whole promo) and returns a percentage score for how much the passage has been written by A.I. The vast majority of promos for the last couple of shows returned low scores (mostly 1% or lower) indicative of original work. Promos from three handlers were flagged as having minor or significant uses of A.I.
Of course, the tool's veracity is something we had little knowledge of (except it being a paid service used by a company, rather than a free online tool), and so we asked those who had used A.I. to come forward. We wanted to know if any others had used A.I. in substantial ways that hadn't been flagged to gauge the system’s veracity. The three handlers that were flagged by the tool all accepted to having used A.I. to varying extents, and the scores are correlated to the extent to which A.I. was used. Talking to those handlers helped ascertain how the use of A.I. looked and what level of input was required to generate the A.I. elements. No other handlers came forward other than those flagged by the tool (other than those who didn't know what we meant by using A.I. tools).
It seems like rephrasing and restructuring tools on Grammarly (or other A.I. systems), as well as automatic synonym replacement tools like Quillbox, were mostly responsible for the high scores. I didn't really know what these things were until earlier this week and I imagine a lot of you don't either. Just as an example of what a rephrasing tool can do, Tommy ran some of his Winter Wasteland promo through the one on Grammarly.
Here is the original, scoring less than 1% on the A.I. detection tool:
Rocco was startled out of his slumber around 4:30 on Saturday morning. Tommy was angrily shoving the Christmas tree that Randi had put up in the living room back into its box. For the first time that he could remember, he was actually looking forward to the holidays, but the nightmares and Keres’ looming presence in the apartment had ruined all of that. He had allowed himself to get excited about the idea of waking up on Christmas morning and watching Randi and Walker open gifts. He didn’t even care that Walker was too young to fully understand what was going on. He finally had the family he had never had before, but Keres had ruined it all.
It had been more than a week since Tommy had seen Randi and Walker. The minutes felt like hours, the hours like days, and the days like weeks. The man who had once enjoyed life as a bachelor suddenly missed the family that he had started. Of course, he wasn’t truly alone. Rocco had set up homestead in Tommy’s apartment the morning that Randi took Walker and fled to her aunt’s home in Tulsa.
And the reworked version, scoring 97% on the A.I. detection tool:
Rocco jolted awake around 4:30 on Saturday morning, startled by the commotion. Tommy was forcefully packing away the Christmas tree Randi had lovingly decorated in the living room. Despite his anticipation for the holidays, his excitement had been overshadowed by nightmares and the ominous presence of Keres in their apartment. He had been eagerly anticipating spending Christmas morning with Randi and Walker, relishing in the joy of giving gifts, even if Walker was too young to fully grasp the occasion. For the first time, he felt the warmth of having a true family, only to have it shattered by Keres' interference.
It had been over a week since Tommy last saw Randi and Walker. Each passing moment felt like an eternity, amplifying his longing for the family he had begun to cherish. Despite the sudden absence of his newfound family, he wasn't entirely alone. Rocco had taken up residence in Tommy's apartment since the day Randi left for her aunt's home in Tulsa, seeking refuge from the turmoil caused by Keres.
You can argue about which version is better but obviously the original is already very good. I imagine the results would be more marked when running something less refined through the same tool, and other tools from other A.I.s may be more or less powerful. Regardless, the difference is clearly noticeable to the point where I don't think the reworked version can be truly considered the writer's own work. Although there is no specific rule in our rules thread about using A.I. in this way, we feel there is an obvious expectation that work put out in a competitive setting is your own.
Although we do not intend to publicly name those handlers concerned, we will reach out to handlers who have lost to a promo that used A.I. in a significant way, such as the use of rephrasing/restructuring tools. This is not a large number of people. We will discuss with them if there are any ramifications for their character. I also just wanted to make clear that no mod character is involved in this as an opponent (successful or otherwise) of a promo that was flagged as having A.I. I should also point out that this is completely unrelated to Cap’s decision to step away from the fed. Cap’s promos were not flagged as having used A.I. in any way and Cap is welcome back to the fed if he should ever choose to roleplay here again.
Going forward, we would like to establish what is considered okay and what isn’t:
- Use of a standard spell/grammar checker, like those found on Google Docs or Word, is fine.
- Use of A.I. to generate images used in promos is fine, as is sourcing images from elsewhere on the internet.
- Use of rephrasing/restructuring tools, such as those found on Grammarly, or using an A.I. tool to rephrase or restructure your promo, in a way that alters the writing in a meaningful manner is not allowed.
- Use of automatic synonym replacement tools like Quillbot is not allowed.
- Essentially, you have to write it all yourself.
You can reach out to a mod if you have any questions or thoughts or concerns you’d rather share privately.
Thanks
Mods
