w  w  w  .  s  c  h  u  m  i  n  w  e  b  .  c  o  m

          

22 minute read

November 1, 2024, 11:55 PM

Recently, a former colleague of mine shared a post on LinkedIn by Benjamin Stein (no, not that guy) that read, “Go ask ChatGPT: ‘Based on our previous interactions, what do you know about me that I may not know about myself?'”  My colleague said in her post that she was using ChatGPT to help with a job search, and shared some of the things that the bot said about her.  Not bad.

I responded to her post about my own experience with ChatGPT, saying, “All I know is that ChatGPT knows exactly who I am, probably because of my large online presence, and it told quite a few whoppers about me,” and then cited the Journal entry that I wrote about it last spring.  My colleague responded that she loved the entry, but considering that the original entry is now more than 18 months old, as well as the rapid pace of advancement in this kind of technology, I should run it again to see what it comes up with.  I liked the idea.

So I ran the entire process over again, asking the exact same questions, i.e. “What do you know about Ben Schumin?” and “Tell me about The Schumin Web,” and running each inquiry five times, using the default model, GPT-4o.  My methodology for scoring each of the responses was exactly the same as before, counting the number of factual claims, and then determining the accuracy of each one.  Accurate claims scored a point, inaccurate claims scored no points, and a mixture of accurate and inaccurate information scored half a point.  Then take that number and divide it by the total number of factual claims made, and that’s the final score in the form of a percentage.  As was the case before, I still don’t know what an expert in this sort of thing might do to rate the accuracy of these responses, but this is the best that I could come up with, plus this is the same method that I used before, giving us apples-to-apples results.

I started out with the “What do you know about Ben Schumin?” prompt, and here’s the first one:

Ben Schumin is a photographer, blogger, and documentarian known for his photo essays and detailed documentation of urban environments, everyday places, and social movements, particularly in the Washington, D.C., area. His work often includes images and commentary on transportation infrastructure, retail spaces, and architecture. Schumin has been active in documenting public events, protests, and city life for many years, especially through his website, The Schumin Web, which he launched in 1996.

Schumin’s style is notable for capturing the everyday aspects of life, often with a focus on locations and objects that may not always be considered photogenic or significant. He has documented significant protests, particularly in D.C., and has contributed photography to Wikimedia Commons, enhancing the free media repository with his images of metro stations, urban areas, and political events. His work provides insight into urban spaces and events, offering a unique lens on both common and unusual aspects of contemporary life.

I see six factual claims here:

• Photographer, blogger, and documentarian known for photography, largely in the DC area
• Focus on transportation infrastructure, retail, and architcture
• Schumin Web founded in 1996
• Focus on capturing things that may not necessarily be considered photogenic or significant by many
• Documentation of DC-area protests
• Contributor to Wikimedia Commons

The first claim checks out.  That is exactly what I do, though lately, my photography has been more outwardly-focused, as I travel a lot more than I used to.  The second claim also checks out, as that tends to be what I focus on as I scroll through the last year or so of published works on my photo site.  They also got the website’s start date correct, as it began in 1996.  As for #4, I can’t disagree.  I remember that I used to get some criticism early on for “why are you photographing that?” but nowadays, those are the photos where I see a lot of downstream usage.  My old Walmart photo set from 2000, which was done mainly to get used to the camera and how it worked and all, and then ended up providing enough content to make something out of it, was a particular target of criticism early on, i.e. “Why would you want to photograph your Walmart trip?”  So that gets a check.  The fifth claim is true, though I have not photographed a protest in more than a decade at this point, as my interests and availability have shifted (Food & Water Watch really killed a lot of my enthusiasm for activism).  And the sixth claim is true, though I no longer contribute directly.  There is a dedicated category on Wikimedia Commons for my old Creative Commons work, and when I uploaded a bunch of older material to my photo sites in 2020-2021, someone else transferred a bunch of them to Wikimedia Commons and tagged them accordingly.

So they got six out of six claims correct.  That’s a perfect score.

Let’s see how the second one compares:

Ben Schumin is an American writer, photographer, and internet personality best known for his contributions to urban exploration, photography, and documenting public transportation. His work often captures elements of everyday life, public transit, retail environments, and urban landscapes, particularly in the Washington, D.C., area and throughout the Eastern United States. Schumin’s photography style is noted for its focus on candid, often quirky, scenes and details of urban life that might otherwise go unnoticed.

He is also known for The Schumin Web, a personal website he started in 1996, which covers his photography, blogs, and various projects. Schumin’s documentation style, along with his emphasis on capturing authentic urban experiences, has made him a respected figure within the niche communities of urban exploration and transit enthusiasts.

Is there something specific you’d like to know about his work or interests?

Here are the factual claims:

• Writer, photographer, and Internet personality
• Known for urbex, photography, and transit
• Focus on daily life, transit, retail, and urban landscapes, mostly in the eastern US
• Schumin Web founded in 1996, covering photography, blogs, and whatever else

I deliberately omitted “Schumin’s photography style is noted for its focus on candid, often quirky, scenes and details of urban life,” and “Schumin’s documentation style, along with his emphasis on capturing authentic urban experiences, has made him a respected figure within the niche communities of urban exploration and transit enthusiasts,” because I considered them more opinion statements than factual statements.

So out of the four claims, all are accurate.  That’s another 100%.

Now let’s see what the third roll of the algorithm turns up:

Ben Schumin is an American photographer, blogger, and content creator known for his work documenting everyday life, public transportation systems, and various urban spaces, primarily in the Washington, D.C. area. He has managed The Schumin Web, a personal website and blog, since the 1990s. Schumin’s blog covers a broad range of topics, including urban infrastructure, protests, travel, and personal experiences. His photography is often centered on cityscapes, transit, architecture, and public events, capturing the essence of city life and the unique details of urban environments.

Schumin’s work has gained attention due to his focus on public and protest photography, particularly during demonstrations in Washington, D.C. He is also known for his meticulous documentation style, often capturing extensive details about public spaces and their interactions with people, making his blog a resource for those interested in urban studies, social movements, and transportation networks. His contributions are notable for offering a ground-level view of significant and everyday happenings alike.

And here are the claims:

• DC-area photographer, writer, and blogger documenting life, transit, and urban spaces, largely in the DC region
• Schumin Web founded in the 1990s
• Said website covers infrastructure, protests, travel, and various life experiences
• Photography centers on cityscapes, transit, architecture, and events
• Gained attention for protest photography in DC
• Meticulous documentation style, with extensive details about the things that I cover
• Ground-level view of significant and everyday happenings

So this run makes seven factual claims.  I deliberately omitted “making his blog a resource for those interested in urban studies, social movements, and transportation networks,” because that is something that I can’t necessarily verify.  The first claim is correct.  For the second claim, note that it casts a much broader net than the previous two, giving a decade range rather than an exact year.  So that’s kind of a copout,  if you ask me.  However, the correct date is within the stated range, so I’m going to rule it correct.  The third and fourth claims kind of go together, and that is indeed what I tend to focus on.  It is worth noting that this has never been an intentional, stated focus, but rather, it’s the sort of stuff that I just tend to gravitate towards.  That #5 claim, meanwhile, amused me, because the first thing that came to my mind was when my coverage of the January 27, 2007 protest was featured on conservative pundit Michelle Malkin’s blog, which led to a firestorm of criticism directed at me, and which I discussed in a Journal entry not long after.  Then as far as the claim about my style of documentation, being quite meticulous, I can’t argue with that characterization of things.  Just looking at my recently published Midwest Road Trip photo set, I said this about a photo of the roof of a Walmart store: “The roof of the Walmart in Austintown. Note that only the rightmost part of the building contains skylights. This is consistent with a conversion store, where the original part of the building dates from the period when Walmart built stores with drop ceilings and no skylights, while the grocery expansion contains an open truss ceiling and skylights.”  That’s pretty meticulous right there.  I suppose that stems from the way that autistic people tend to infodump, and my documentation style fits that mold.  And then as far as the ground-level view of life, it’s not inaccurate.

So we have seven claims, and they are all accurate, with a score of 100% across three rolls of the algorithm so far.

Here’s the fourth one:

Ben Schumin is an American writer and photographer known for his contributions to Wikimedia Commons and for documenting everyday life in various places through photography. His personal website, The Schumin Web, has been active since the mid-1990s, making it one of the earlier personal websites on the internet. Schumin’s work primarily focuses on public spaces, transit systems, and protests in the Washington, D.C. area, where he’s based.

He’s particularly noted for his extensive photo documentation of the Washington Metro, capturing the architecture, signage, and overall transit experience. Schumin has also photographed and reported on various social and political events, including rallies and demonstrations, often with a neutral and observational approach. His contributions to Wikimedia Commons have made his images widely available for public use, often used in articles and projects related to D.C., public transit, and protest documentation.

Here are the claims:

• American writer and photographer
• Known for contributing to Wikimedia Commons and documenting every day life through photography
• Schumin Web has been active since the mid-1990s
• Focus on public spaces, transit, and political activism in the DC region
• Based in the DC region
• Noted for extensive documentation of the DC Metro system
• Photographed and reported on social and political events with a more observational approach

All of these claims, save for the last two, we have already examined, and have established that they are accurate.  Note that while it gives a range for when this site began, it is a smaller range than #3.  On #6, about the Metro, that’s a first for this exercise.  I have documented the Metro system pretty extensively, having photographed every single station in some way or another.  Not all of those photos are necessarily still available online, mainly because of the gradual evolution in how I present my content as some stuff is added and some stuff is removed, but it exists.  So that is accurate.  As for #7, I was on the fence about whether to include it as a factual claim or omit it as an opinion, but ultimately decided to include it as factual because that’s how the system presented it.  And admittedly, while I never thought of it that way, I can’t necessarily disagree with it.  After all, when I went to protests, I wasn’t the one making any calls to hijack a march or head to the front of the Capitol.  I was just photographing.  I also know that I certainly have my biases on things, but apparently I’ve made someone think that I’m a neutral force.

So I suppose that we have another perfect score.

Here’s the fifth and final run with the first inquiry:

Ben Schumin is known for his personal website, The Schumin Web, where he shares photography, essays, blog posts, and other content documenting everyday experiences and observations. His site, which started in the 1990s, features a wide array of topics, from public transportation and urban exploration to retail history and personal reflections. Schumin’s work has gained attention due to its thorough and detailed photographic records, especially of the Washington, D.C., Metro system, urban landscapes, and retail stores, including now-closed department stores and malls.

His content is often noted for capturing the ordinary in an engaging way, and he has documented everything from protests and public events to abandoned buildings and shopping malls. Schumin’s work appeals to those interested in urban life, transportation, and a nostalgic look at retail culture.

Are you interested in any particular aspect of his work?

Here are the claims:

• Known for Schumin Web with all of the content that it has on it
• Site started in the 1990s
• Focus on the DC Metro system, urban landscapes, and retail
• Inclusion of now-closed malls and stores

Yep.  The first three are rehashes of what we’ve already determined is accurate, and this is no exception.  The fourth one, I like, because it notes that I have covered retail subjects that no longer exist, like Staunton Mall or the Kmart in Bridgehampton.  I don’t know when ChatGPT was last trained, but that’s pretty astute, especially as I’ve gone back to Staunton Mall a number of times in the last few years to provide more updates (and if all goes well, look for another Staunton Mall update in December).

So as far as that goes, I believe that we have another 100%.  That’s five for five in describing me.  I was not expecting a perfect score across five runs.  I was expecting some inaccurate content to make its way in there.

With the five queries about me completed and scored, and a perfect score across the five (a tremendous improvement over the 50% score from March 2023), let’s go on to my other query: “Tell me about The Schumin Web.”  I admit that asking it about me five times got old because it was so accurate, so I was ready to move on to a slightly different subject.

Here is the first cycle:

The Schumin Web is a long-running personal website created by Ben Schumin, originally launched in 1996. The site serves as a comprehensive online journal and photo repository, featuring Schumin’s personal thoughts, photo sets, essays, and commentary on various topics. It began as a simple site documenting his college life and interests but evolved into a more structured collection of content over the years.

The site’s major sections include photo essays, which often capture urban life and places Schumin has visited, as well as writings on public transportation (especially the Washington Metro), retail (like the rise and fall of major department stores), and protest culture. Schumin often updates the site with new photos and reflections, and it stands out for its very personal tone combined with an emphasis on documentary-style photography and observations on everyday life. It’s also notable for surviving the shifts in internet culture over the past few decades, maintaining its place as a niche, content-rich personal blog.

Is there a particular part of The Schumin Web you’d like to know more about?

Here are the claims:

• Long-running personal site launched in 1996
• Journal and photo repository, featuring thoughts, photo sets, essays, and commentary
• Began as a simple site documenting college life and interests
• Evolved into a more structured collection of content over the years
• Major sections include photo essays, writings about transit, retail, activism
• Stands out for personal tone with documentary-style photography and observations about everyday life
• Survived various shifts in online culture

The first claim is accurate, as it’s been running for more than 28 years at this point, initially launched in 1996 on members.aol.com (how’s that for old?).  Second claim is also accurate, as it describes the content nicely.  With the third claim, I finally find a mistake somewhere, as it says that it began as a documentation of college life and interests.  My site actually goes back earlier than that, as I started it as a freshman in high school.  The site was already well-established by the time I went to college, and it continued its gradual evolution all throughout college.  The main difference with my college years was that was when it started to get some attention outside of my local sphere, as it was featured in Front magazine, on SpinnWebe, and in various other locations.  So while someone could be forgiven for thinking that I started the site in college, especially considering that I didn’t really archive content in those first few years, it is all the same inaccurate.  So I’ve got to ding them for that one.  The rest of that sentence, where it states that it evolved into a more structured collection of content over the years, is accurate, as the site was just kind of a blob of content until 2000, when sections were first introduced to better organize everything as the site grew.  Schumin Web is just a little bit older than the service thinks it is.  The description of major sections is accurate, as that captures my focus pretty well, though there is so much more that I go over, like AI chatbots.  The sixth claim, that says that it stands out for its personal tone and documentary-style photography seems to track, especially looking at various photo sets and Journal entries over the last few years.  And that seventh claim, where it says that it has survived shifts in online culture, is true.  I mean, Schumin Web outlived ICQ, for one thing.  It was around before ICQ started, and is still there now that ICQ has gone.  It has also outlived AOL Instant Messenger, for that matter.  It’s also older than MySpace, Facebook, Twitter, YouTube, Google, Wikipedia, and a whole host of other services that we now consider the backbone of the Internet today.  I was here before all of them, and the whole time, I’ve just been doing my thing on my little virtual island.

So that’s six accurate claims out of seven total, for a score of 85.7%.  If I were to assign a letter grade to that, it would get a “B” based on the standard ten-point scale.  Not bad.

And here’s the second one:

The Schumin Web is a personal website and blog created by Ben Schumin, who has been running it since 1996. The site originally began as a college project and has evolved into a long-standing online journal and archive, covering a diverse range of topics from public transportation, urban exploration, and retail store histories to personal reflections, photography, and current events. Schumin’s content is often known for its unique documentation of everyday places and events, particularly in the Washington, D.C., area.

One of The Schumin Web’s most notable features is its extensive photo documentation of public transit systems, particularly the Washington Metro, and his detailed accounts of protest events, especially in D.C. Schumin captures these moments with an observational style, offering both descriptive and occasionally humorous commentary. The blog’s commitment to detailed photography and historical context has attracted readers who are interested in niche subjects like mall architecture, urban infrastructure, and the local culture of the mid-Atlantic region.

The website itself has seen various redesigns over the years, yet it maintains a retro, personal-blog feel that aligns with its 1990s origins. It’s a classic example of an independent web space that’s persisted through decades of internet changes, staying true to its creator’s interests and personal storytelling style.

Claims are as follows:

• Ben Schumin’s website and blog
• Started in 1996
• Originally began as a college project
• Evolved into a longstanding journal and archive
• Covers transit, urbex, retail, personal reflections, photography, and current events
• Documentation of everyday places and events, mostly in the DC region
• Observational style with descriptive and humorous commentary
• Various redesigns over the years, but maintains a retro-personal-blog feel reminiscent of its nineties origins

First of all, regarding that last line, where it says, “It’s a classic example of an independent web space that’s persisted through decades of internet changes, staying true to its creator’s interests and personal storytelling style,” I am leaving that alone, since it’s more of an opinion statement than a factual claim, though I appreciate the sentiment.

Regarding the various claims, I’ve already gone over the first two with similar claims in other places, and these were all ruled accurate.

However, it fails again on the third one, where it claims that it began as a college project.  Not just began during college like the other claim did, but specifically as a college project.  And it’s still inaccurate.  I started it on my own just to do it when I was in high school.  However, I have used the website as it existed at that time as a college project before with permission of the professor, and got an “A” on it.  That project was for a summer class where we were required to make a website.  I already knew how to do that, and had been running it for six years at that point.  I had therefore already demonstrated mastery of the subject, and everyone was happy about that – particularly me, because it meant a free “A” for something that I was already doing.  I remember when we did a similar project in high school, when everyone else was making some dinky little personal website, I was doing a redesign of the school’s website for them in the class in coordination with the school’s administration.  Said website actually went into production not long after I graduated, and if you look at the Internet Archive, there is my website for Stuarts Draft High School, complete with my photo of the school’s original sign on it.  But for purposes of this claim, though, I’ve got to ding them once again for getting the origins wrong.

The fourth claim is true, about the site’s evolution, as are the fifth and sixth claims about the subject matter.  Claim #7 is fair enough, and I’m glad that someone, even if it’s an AI chatbot, appreciates my jokes.  So I’m ruling that as accurate.

For the eighth one, I say yes, but it’s not an intentional decision.  Rather, my skills as a web designer have their limits, and those limits sometimes show.  I also admit that it’s been too long since I’ve last redesigned, with the current design’s coming out in 2012.  I would love for the site to do a lot of different technical things that it doesn’t do now, but I just don’t know how to implement them.  I have also always avoided having other people do the design work on the site, because I like knowing all of the ins and outs of it, such as where the code may be inelegant or incomplete, and if someone else did it, I won’t have that intimate knowledge of how it all goes together.  The photo portfolio site is entirely third-party (I am only responsible for the content there), which is why it looked so ugly at first, though this has now mostly been smoothed out with some experimenting with the various themes over there.  Not everything over there is how I might want it, but it works pretty well.  By comparison, on this site, even after transitioning over to WordPress in 2012, I have always built my own interface.  It may not be the most modern or the most technically sophisticated design, but it is entirely my own work, and if something doesn’t look right or doesn’t work right, I know where to look in order to fix it.

So with that in mind, eight claims in total, with one wrong claim gives us an accuracy of seven out of eight, or 87.5%.  Still a “B”, which is reasonable enough.

Third one:

The Schumin Web is a long-running personal website and blog created by Ben Schumin. Established in 1996, it is one of the earliest personal websites to continually exist on the web. The site covers a wide range of topics, reflecting Schumin’s diverse interests, including photography, social commentary, personal anecdotes, and explorations of urban areas, especially around the Washington, D.C. area. Schumin also documents trips, protests, and events he attends, often with a focus on public transportation, retail history, and the urban environment.

Ben Schumin is known for his candid style, capturing everyday moments and providing his personal take on current events and cultural trends. Over the years, The Schumin Web has grown from a simple web project to a substantial archive of content, serving as a digital scrapbook of Schumin’s life and experiences. The site remains active, regularly updated with new posts and photos, and continues to reflect changes in internet culture and web design, making it an interesting case study in the evolution of personal websites.

And here are its claims:

• Ben Schumin’s long-running website established in 1996, and one of the earliest continually running personal websites
• Subject matter including photography, social commentary, personal anecdotes, and urbex, with a focus on the DC area
• Documentation of trips, protests, and events, with a focus on transit, retail, and urban environment
• Candid style, capturing everyday moments and providing my personal take on it
• Grown over the years from a simple website to a large digital scrapbook
• Site still active, and reflects trends and changes

I admit that I liked that last bit, where it says, “making it an interesting case study in the evolution of personal websites.”  It’s an opinion statement, so I can’t score it, but, hey, at least something finds me interesting, even if just as a case study.

As far as factual claims go, this is quite straightforward: yep, yep, yep, yep, yep, yep, now take your perfect score and get out of here.  Though I always love it when ChatGPT recognizes the site’s longevity.  How does an AI chatbot even determine that?  I would love to know.

And onwards to number four:

The Schumin Web is a long-standing personal website created by Ben Schumin, who launched it in 1996. Originally started as a hobby website, it has since grown into a diverse collection of content that includes photography, blog posts, personal stories, and commentary on various subjects ranging from urban exploration and public transportation to retail history and day-to-day musings.

Ben Schumin has made The Schumin Web notable by focusing on topics he’s passionate about, including an extensive exploration of Washington, D.C.’s Metro system and the photography of protests and demonstrations. His writing style is approachable and often reflective, sharing insights and observations on his experiences in an informal yet informative manner. The site has amassed a wide array of photo essays, narratives, and opinion pieces that have made it somewhat of a personal archive, documenting his interests and moments in his life.

Since The Schumin Web has been around for so long, it also serves as a kind of time capsule, reflecting both the evolution of personal web publishing and Schumin’s own interests over the years. While it’s a niche site, it’s gained a small but dedicated audience who appreciate the eclectic mix of content and Schumin’s unique, individual perspective on the world.

The claims are as follows:

• Longstanding personal website by Ben Schumin that launched in 1996
• Started as a hobby website
• Grown to include photography, blog posts, personal stories, and commentary on many subjects, including urbex, transit, retail, and whatever else I find interesting
• Extensive exploration of the DC Metro system and various activism work
• Has amassed a lot of stuff, becoming something of a personal archive
• Its being a long-runner causes it to serve as something of a time capsule, reflecting both my own evolution and that of wider trends

And again, I have to leave out my favorite statement, where it says, “While it’s a niche site, it’s gained a small but dedicated audience who appreciate the eclectic mix of content and Schumin’s unique, individual perspective on the world,” because it’s an opinion statement, but I do find that rather insightful.

As far as these claims go, the first one is accurate.  The second one impressed me because it actually gets it correct that it started as a hobby website.  The next three are nothing that we haven’t seen the chatbot discuss before, and it is all accurate.  The last one is accurate, though unintentional.  Someone commented to me after I said that reading some older parts of the website make me kind of cringe that it means that I have grown and evolved as a person in the years since those various posts were made, and I can’t argue with that.  I also make specific effort not to rework older material in order to keep it as a product of its time as much as possible.  After all, my writing style has changed in all of these years.  Journal entries have gotten longer and more detailed, and the way that I cover things has changed.  I used to be able to bang out a Journal entry in an hour.  Now, most entries take multiple sessions to finish, especially photo-heavy ones like the weekend trip to Charleston that Elyse and I did in April, or that weekend trip to New York and New Jersey that we did with Kyle in September.  The treatment is also different.  Compare this Journal entry from 2007 about a trip that I made to DC with a friend, to a recent short adventure in Pennsylvania.

So we have another 100%.  ChatGPT is doing markedly better this time compared to March 2023.

Then to round it all out, here is the fifth and final go at it about Schumin Web:

The Schumin Web is a long-standing personal website and blog created and maintained by Ben Schumin. Established in 1996, it covers a broad array of topics, reflecting Ben’s diverse interests, including personal stories, photography, urban exploration, politics, and observations on daily life. The site is often recognized for its early roots in the internet blogging space, evolving over the years from a static homepage into a dynamic blog, making it one of the internet’s more established personal websites.

Key sections of The Schumin Web include photography collections, writings, and specific interests like public transit and infrastructure. Ben’s explorations of Washington, D.C., and his interests in social issues, including coverage of protests and public events, have also been featured prominently. The site has gained a small but dedicated following for its genuine tone and the eclectic mix of topics.

The design has gone through many transformations over the years, keeping up with changes in web design trends while retaining a distinctly personal and nostalgic feel.

The factual claims are as follows:

• Ben Schumin’s personal website that’s been around for a long time, established in 1996
• Topics include all of my various interests, with personal stories, photography, urban exploration, politics, and other observations about life
• Evolved over the years from a static homepage to a dynamic blog
• Sections include photography, writings, transit, and infrastructure
• Explorations of the District of Columbia, and interest in social issues to include protests and such are featured prominently
• Site has transformed many times over the years

Like other cycles, the comments that I like most are ones that I can’t really evaluate for accuracy, such as, “The site is often recognized for its early roots in the internet blogging space,” and “The site has gained a small but dedicated following for its genuine tone and the eclectic mix of topics,” since those are both opinion statements.

As far as the claims go, yes, yes, yes, yes, yes, and yes, though with that sixth claim, I’m giving it the benefit of the doubt because while I’m not satisfied about how it’s kept up with design trends.  While my WordPress theme is very dated from a technical standpoint, the site has kept up trends in other ways, such as adopting and getting rid of framesets, and various other little niceties that have been added and/or have evolved over the years.  One example of this is clickable images on the site.  As originally designed, those small images were all that were displayed.  Then in February 2002, I made those small images clickable, adding the full-size images behind the original smaller ones, going from this small size to this much bigger size.  And the way that I set it up to display those was so unnecessarily complicated back then with a separate database just to power those photos, in order to provide a branded page around it with a caption.  Nowadays, I have a WordPress plugin that provides a lightbox for it, and I connect it all with just a simple link.  And if anything should happen to the plugin, it falls back to just a link to an image.

So in scoring this, it gets 100% again.  So for the “Tell me about The Schumin Web” query, it only had two misses across five runs.  That gives it an overall score of 31 accurate claims out of 33 total factual claims, for a score of 93.9%.  That is an “A”, and a big improvement over the 80% that it got in the earlier run.

Looking at it overall, I have gotten the sense that ChatGPT has improved its algorithms considerably.  With all of the claims made by all ten queries, it made 61 total factual claims, with only two misses.  That’s a final score or 96.7%, which is in the “A” range.  I have no idea what sort of work or methodology goes into making it, but I run test queries and then remark on what I see.  One thing that I noticed this time around was that it had become much more conservative about what it would output than it did before.  When I asked it about me back in March 2023, it mentioned my date and place of birth in four out of the five responses, and not a single one of those four responses was accurate.  Likewise, in the earlier run, it mentioned where I attended college in two of the responses, and both were very wrong, placing me at Virginia Tech and the Rochester Institute of Technology.  I’m sure that they’re both lovely schools (my sister went to Virginia Tech), but I did not go to either one of them.  The new run does not mention my education at all.  Similarly, no mention of any awards (because there aren’t any), and really nothing about me of any kind outside of the context of the website.  I wonder if it now has a certain confidence threshold that must be met before it will output something, much like Watson did on its Jeopardy! matches.  That might explain why it didn’t output my education, my date of birth, and my place of birth, because it didn’t have enough confidence in the information that it had to commit to a response.

Additionally, I noticed that most of the responses tended to agree with each other.  They all mentioned the origin date of the website, with seven of them correctly giving 1996, and the other three giving a range which contained the correct date.  Additionally, just about all of them emphasize my coverage of transit, urban exploration (even if I think that the responses place too much emphasis on it), activism, retail, and architecture.  The only outright factual errors that it made in all ten cycles were about the website’s origins, i.e. how it started.  Everything else was accurate, even if I wouldn’t necessarily describe it that way, myself.  If it tells you anything, it was uniform enough that I was tempted to write “and blah blah blah” with some of the factual claims because I didn’t want to say the same thing over and over again, and I also was starting to get bored when writing out, analyzing, and scoring these claims, because they were all so similar to one another.

So I would say that in about 18-19 months’ time, ChatGPT has improved considerably.  I still wouldn’t rely on it as far as I could throw it, and I especially wouldn’t use it for something of any importance like some lawyers in New York did in the spring of last year, but it seemed to provide decent enough information about me and my website when queried.  It’s not amazing, but it also won’t send someone on a wild goose chase looking for birth records for me in Washington, DC.

I finished off the earlier entry by saying that it had a long way to go before being ready for prime time, that it certainly had its moments where it impressed me, and that at that point in time, it was a curiosity more than anything else.  This time, I can say that it has more potential as a utility, though I would verify every single statement that the bot makes against other sources before relying on any information that it provides, especially since the bot still doesn’t cite the sources that it relied on to provide its information.  So it’s still very much a use-at-your-own-risk proposition, but much less risky than before.

Categories: Artificial intelligence, Schumin Web meta, Social media