Balancing Innovation, Human Judgment, and Connection in the Workplace

Jing Hu 

Thank you. I do think it’s going to be a great conversation because I was reading the questions that you sent over and the topics that we are about to discuss is. They are all very exciting topics that I’ve been looking into for the last two years and I am keen to share the conclusion that I got from these research with you and your audience. 

Hessie Jones 

Thank you. That’s amazing. OK. So let’s let’s talk about one of the articles that kind of led me to you and the title was why thinking hurts after using AI. And this is one of the the things that you had said, seduced by AI’s. Convenience. I’d rush through tasks. Sending unchecked emails and publishing unvetted content. So within that article you’re talking about some research that had been done by open AI and New Pen and the the ability for people to actually develop these seemingly amazing. You know, pieces of text within e-mail or social media posts like within seconds. Can you talk about a little bit about that study and what were some of the findings from it? 

Jing Hu 

Yes, it it was actually one of the earlier studies about ChatGPT. I think that study was released in 23 or late 20 or early 20. I think early 20 zero. Sorry cuz that’s only the time when tragedy came out. So what? That study is that what that study was about was talking about the occupations that would be impacted by. By genai like transcript. So it mentions that especially occupations that relies a lot on science and critical thinkings are the occupations that would be safe. But occupations that relies a lot on writing and programming. Or the occupations that would be at risk. It’s kind of aligned with what we have seen on the media. You probably saw a lot of the news saying, hey, this company decided. Not to hire programmers and all of. Programmers being laid off because of AI but. That is the hard side that I am. Keen to get into in. The later part of the conversation, we focus on the rest of the of the rest of the workers in the US it’s it was saying 80% of the US workers could have at least 10% of their work task affected by LLM. 

So this is the. Point that I really want to break it down to you because I think first this study was a while ago about two years for an AI. An AI study is kind of old already. If you combine with what we’ve seen so far and what the study has to offer, there’s still some. Value, especially the methods they pull together. But we need to. Review this study from a higher level in order to see, like, what’s still true and what are the things that we need to rethink. 

Hessie Jones 

OK, I do. Oh, sorry, go ahead. 

Jing Hu 

No, no worries. You were about to ask. 

Hessie Jones 

I just wanted to pull up like a few examples of companies like Duolingo that I believe that they’re one of the ones that got rid of. I’m not sure if they’re entire developer workforce. There is also one other company. Out of California, I forgot their name.  When ChatGPT launched, they basically got rid of their content creators and they started just generating articles directly from it. And in both instances there that they tend to backtrack and say it’s not that good and were a bit presumptuous in doing this. 

Jing Hu 

That’s exactly what I wanted to cover. So. In that study, it mentioned, of course we talked about a little bit like critical thinking and science. Are the skills you need in the future. It also pointed out two skills that will made obsolete once AI becomes more mature. One is writing. And another is programming the argument makes a lot of sense, but if you pull pulled it to today, as you said, a lot of company actually retracts their decision two years. So when it comes to writing, we all know Gen. AI is really good at drafting an article summarizing something and formatting. But this is something people finally started to realize. We humans are irreplaceable when it comes to strategizing. What are we writing when it comes to style, bringing the personal touch into the article you write. It or when we talk about things that’s really domain specific because I only know about generic knowledge and when it comes to originality like. For example. When we’re crafting a marketing line, we thought AI could help us do all the copywriting and we no longer need writer. But that’s not true, because you started to realize the repetitive words used from AI. It would use something. Like it is not about something, but it is about something. These kind of sentences or it will give you a lot of em dashs or a lot of common like that. That’s the kind of kind of patterns so easy to pick out and people feel bored of it. So that’s the writing part. When it comes to programming. Yes, AI is good at producing code. 

Again, it’s something that. We’ll talk about later, but like the code itself. Is such a small part in terms of software development, people often ignore the parts that we do need. Architecture, design. We do need to think about security. We do need to configure properly. Cloud setup in order for the whole operation to run. 

Coding is only like 20% of it and most of the codings are done by junior people who just came out of college. They focus on code, but that’s exactly the parts that AI also failed, and they even led juniors to a past that we don’t want them to. But come back to the main point is no writing and programming on being replaced by AI yet if we. Great. This concept down to further you would actually realize there are two parts, one things that will be replaced by AI. And the other one is the part that won’t be replaced. The part that will be replaced by AI will be routine repetitive. Small and not involve any decision making. A lot of pattern recognizing. And very short-term work like copywriting. Those are the things that would still be easily replaced. So that’s the time when we need to think about the long-term stuff. How do you start involving yourself in complex, not routine problem solving in decision making that is ambiguous and hard to make a decision if you unless you go into like a deep dive analysis and analysis that goes beyond the pattern. 

Recognizing and then of course you need to understand what kind of a long term impact your job would do to to your employer so that you would still have that value to be in the company instead of. Like they would think that your work being. Replaced by AI. 

Hessie Jones 

So I want to take some of the learning that you just presented to us and I want to discuss that in the context of. The next article. Because this is it’s. It’s somewhat similar, but this is this is also. About, you know, research, it’s about deep thinking, but this is at the college level. So there’s a recent article from the New York Times. Its article was everyone is cheating the way through college and one of the quotes was, let’s face it, Gen. AI is much more effective of content generation. It’s become good, so good or sorry. Good enough that one student remarks with chat GPT I can write. An essay in two hours. It normally takes 12. If it isn’t, it’s not as if cheating is new. But now one student puts it the ceiling has been blown off and you could resist a tool that makes every assignment easier and seemingly with no consequences. So one professor actually remarked there are massive numbers of students that are going to emerge from universities with degrees. Into the workforce, who are essentially illiterate. So what? Are your thoughts on this? 

Jing Hu 

So I have to say those observation is accurate to an extent. So I I’ve been speaking with a few professors in the last few months. Their observation is mixed. So you do have a group of students who would already cheat. Regardless, AI will not. You do also have another group of students who genuinely want to learn from what the professors have to offer? So for those who would cheat regardless, what it’s probably less of AI, but another part of education that we need to work on, but that is never going to be resolved if we just simply remove AI. 

Hessie Jones 

Can I say I think one thing that I did read about this topic specifically is that the incentive structures also need to change within the education system. So for example in Japan, they put less emphasis on grades. In here in North America, we put a lot of emphasis on grades. So when you learn it’s on you right. And it’s if you’re paying for education and you’re willing to put in the time to actually get something out of it and develop your own knowledge. But if it’s the grade that matters, especially at a point where you’re saying, OK, I’ve had enough of four years of university and I’m ready to go. So maybe my last for law is I’m not going to write this essay. I’m just going to leave it up to the machine. By that point. You you’ve already acquiesced, and you said, you know, that’s it, right? So. So my point about the incentive structure is that it’s like you said, it’s on you. It’s up to you whether or not you want to do it, but does the system have to also encourage? That type of thinking, in order for people. To cheat less. 

Jing Hu 

So this actually one of the topics that we discussed on the panel of Greenwich University, one professor was asking. If there is another way to conduct an exam that’s not writing an essay at home, or that’s not like choosing ABCD or E, which one is the right answer but more around? Like if you can focus on delivering A verbal exam. That’s something that you can’t cheat yet, I hope. More if you can. And focus on the kind of exam that would more about observing students behavior rather than whether they give out a a right or wrong answer. There are a lot of subjects that you can’t really cheat on an event, for example, chemistry. If you have to go through lab. And you have to come out with the correct compound, then you can’t really cheat on that or in some the the physics test is the same. So it really depends on the subject and it’s hard to say though, there are much more methods than we. Thought to conduct an exam. 

Hessie Jones 

It’s I think for the most part, I don’t know where if in the US or the UK this is the case, but I know in Canada a lot of our students  in cases will write their exam, like in handwritten form. So, they don’t have unless it’s math you have that access to a calculator, but for the most part you. Have to write it so during COVID I know that changed because people couldn’t congregate and so that became more important to be able to do exams remotely and digitally. But I think the part that that’s more. Relevant here is not, not necessarily the exams, but actually the projects and the essays and whether or not even the professors can distinguish. You know whether or not it was produced by a student or enabled through these generative AI technologies. 

Jing Hu 

Yeah, that’s an interesting one. And we can probably discuss that for the rest of this afternoon. Just one point there was also another interesting interesting question raised on that panel was. I’m aware I’m not answering your question. Well, I’m answering your question with another question. Well, that professor was asking was if there is a student who aren’t native English speaker, but the exam require the students to write in. In English, to deliver the essay, would you prefer this student to write in poor English but with entirely their own thought? Would or would you prefer the students to feed their prompt into AI and then in the end copy paste what I wrote but still written in an English that you can immediately smell that’s written by it. Which one would you prefer? So it’s not really a question for you because what happened in the end. No one, no professors. On that panel could answer this question. It’s just so difficult and such a dilemma that no one can really make a decision at that point and it’s going to be hard. It’s going to be hard to. For our education system to really be fair and for them to even decide what is fair. So like. 

Is going to Be a huge impact. Well, there is already a huge impact on the education system, a lot of things not decided yet and hard to make a decision on unless we have like. Unless technology evolve again, so maybe we don’t. I even need to think about. It. Or unless there is going to be a standard or unified answer. To those questions. 

Hessie Jones 

I actually think that you brought up a good use case when it comes to language because that. Is an amazing use case to be able to use AI to help bridge that gap. But again, it it’s never been done before and to what extent Is it accurate and it accurate not only in the translation, but also in the context in the way that the individual meant for that answer to be it it it’s. I mean let’s throw it. Throw it up in the air at this point because unless it’s never happened, right?  

 I’m going to morph a little bit because I’m going to take this back to the workplace where we’ve seen an explosion of generative AI. There is much more streamlined, you know, outcomes. People are using it way more often. People are generating social media posts like like you’ve indicated, content is becoming one of those things that it can do really, really well and. You know, for the most part, industry is saying, yeah, let’s do it. Like, why? Why do we need to overthink some of this stuff? But? You’ve also talked about the fact that one thing that the technology can’t replace is research. It’s the amount of collaboration, the, the deep thinking. So tell me about tell me about that from the research that you’ve done, your personal experiences and how the world is changing and what can we? Expect from it. 

Jing Hu 

So I can really help a lot in the workforce. It can help with storming. It can help you write text. It can help you summarize in what happened in the meeting if you. Weren’t. If you weren’t. Really listening in the meeting, you only sit there. Like that’s it and I can’t help. But whatever AI can do, it’s going to be useless. And it could even to the point of being counterproductive in brainstorming or in writing if you don’t have the clear direction yourself. The worst case is that someone used AI but lost track entirely to it. Like in this research I read that there are just a lot of cases about how people started using AI with good intention and wanted to improve productivity. But in the end that they lost track. To it, they didn’t even. Remember what they were about to do. At the beginning. So this happens so often because it takes self-confidence and it takes really good AI literacy. The understanding of the limitation of AI or something I’ll call AI’s traits. How much of AI’s traits. Do you know in order to work with or work around it when necessary? I the lesson I learned is. What is important for everyone to understand is. Find out what is the most optimal time for you to intervene with. Intervene the things that AI is doing. Instead of anything else, it’s just find the point in your workflow in your workflow that you are going to. Start doing the task on your own without the help of AI to make sure that you don’t get pulled away by AI’s output. 

Hessie Jones 

It’s it’s funny because it doesn’t take long to almost develop this addiction. I don’t know if you found that I literally used it when it came out. Then I could bang out like five articles within a week and I just inputted my transcript and I spoke. Give me an outline and then instead of writing based on that outline, I said OK, generate another article based on that outline and the thing that I found is it produced something. OK, I’d have to go in because like, it started generating its own conclusions, which I didn’t want it to do. I just wanted to write the Facts. And so I think yes, it took me less time to actually write and do and do an. Fine, but it took me a whole hell of a lot of time to just re edit everything and then restructure it in a way that that that was fitting with my style. So it and also I lost my my ability to really think of simple words. So I wanted to ask your experience, because you’re a writer and to what extent did you decide when to intervene? 

Speaker 2 

So very difficult question. So I’m a writer. I was also a proud lead in software industry. So one thing that I really can’t help myself doing is to keep trying different kind of AI tools so that I will find a better one that will fit into my workflow. In the end I realized it’s not really about it. The tool. It’s really about how I can slowly discover and construct my workflow that makes the most sense to myself. So I normally start with. Like the idea is always mine, so I normally start with an idea of what kind of research am I going to. Am I going to look out for or a lot of time like I got more and more people throwing papers and studies at me for example than you also throw me a lot of articles and studies. And I I just have like, stuff piled on to read through. I realize those are always. Really interesting entry points. So I don’t rely on AI when it comes to idea generation. 

Normally when I have an idea, I will throw my idea to AI, get AI to ask me some questions. And that’s when I started to formulate. The outline of the article, or whether. Whether I choose this direction to go down to or should I just go with a complete different study or different? Research and then I would rely on AI to help me clean up my draft though it to an extent. It’s somewhat annoying because there are times. Well, actually, most of the time I would like to write in a way that I. In the way I talk. But I will add it out. It would want to achieve the perfect grammar that it deemed a perfect grammar in American English. So it will edit out. A lot of my talking to my audience and lot of like the, the OR some like 🙂 I I don’t really do emoji in the sense. Only do like characters so it will edit that out. So in the end same as you. Like few months back, I would have to edit all of those back, but now I would make the point of intervening much earlier, started to edit my own thing and then in the end only use Grammarly to check so. In my workflow, I don’t use AI that often, even though I talk a lot about AI, the most often used probably is. I use AI to help me generate image that would be the cover or use AI to help me generate my audio for my podcast. 

Hessie Jones 

Yeah, that’s interesting. That’s the the one thing I. Will say I. I started cheating. I I started saying change all the words in here to reflect. You know how all the connecting words like she stated or he surmised. Or, you know those kinds. Upward. They’re so repetitive that I I can’t find different words to actually say, stated asserted. All that stuff. So so I’ll have the LM go through the article and I’ll just say, can you replace them these words? So it becomes it becomes less you know. Repetitive from that perspective, right? And then I realized that. It’s doing the work faster than me, but I really, I really need to do it. I need to go into that dictionary that thesaurus looks through a word, look for alternatives, because that’s the way I did it before. And it made me less it made it. It makes me more lazy if I do it the other way. So that’s what I’m finding.  

So let’s delve into this, because the MIT report that was that I that was one of the research pieces I threw that talked about the cognitive decline in our AI systems. I want to. I want to talk about that because they actually talked about the, let me see the use of encephalogram IMS to monitor the brains. And they they were starting to show that these LLM’s actually provided a view that that people. Can’t even remember one of the quotes that were used in their writing. Can you can you talk about that from your perspective because now they’re actually measuring brain activity and they’re talking about level of engagement and cognitive impairment over time. 

Jing Hu 

It it probably was only the truth of term that you mentioned cognitive decline. In fact it should when we talk about it we will talk about cognitive offload because decline means that it will it’s some long-term impact impact but in the. So the you don’t really see the long term impact because it only has like four months, four months research in between though that said I think your assumption may be right because. One of the theory in how humans form amnesia is that is the lesser brain you use. Of course there is some kind of protein like better sheet folding in your brain and that cause some kind of a channel book. Part of that was. The lesser of the brain function you use, the more likely later on you would develop a disease of forgetting and not remembering and never be able to like recite anything so that itself. Could be a really interesting research I I would love to find a researcher that would be able to do this kind of a long term research, but I imagine it’s going to be very expensive because then you need to monitor people from now on, find a group from now on and continuously monitor them for like 10 to 30 years time. Whether they use AI will not use AI, and that control group is going to be difficult, but I think it could be a potential likely consequences of cognitive decline, so the offload. Sorry, I was being really nerdy. The offload that we’re talking about. Is interesting. I would like to talk about one phrase first. 

I think you would. Likely heard of this phrase is that AI won’t replace humans, but humans using AI will. 

Hessie Jones 

Interesting. Yes, I have think. 

Jing Hu 

I think this is somewhat ******** because it’s. The kind. Of motivational lines about AI that would push a lot of people use AI. But if we come back to. All those studies about AI and critical thinking that released in the last year you would start to know it’s less about. Whether we use AI or not, but it’s more about. Ourselves, It’s always about Yours or your audience mindset determine like how you would interact with AI or how you would leverage AI and then consequence consequently. That would also affect whether you will offload your cognitive load to AI so you. Can sit back and be and slack. So. There are few studies. That I can talk about. I don’t know if. You want me to? Like. That’s what we for. So this is a list of concepts I pulled together which I called AI personality or AI traits. It’s the combination of the of those studies I read between how AI would really impact. Our human behavior. Start with Those with low. 

Self-confidence will rely more on AI and further reinforce their reliance on AI. 2nd is those who. Weren’t so good at learning already will lose track to AI in whatever activity they do. 3rd is the help from AI is going to replace the support from friends. And then AI has 80% higher chances of being more persuasive than humans. And then the last one I pulled from the last study, the MIT one we talked about is that. For someone who doesn’t use their own brain first, just use AI right away for information gathering or even for writing stuff tend to not have the long term memory about their work and also don’t have the ownership. The feeling of ownership about their work. So these five AI characters AI traits are the things that we need to know, and we need to be aware of to make sure when we. Work with AI on a day-to-day basis. We don’t lose our critical thinking to it. We don’t. Stop. That we shouldn’t stop judging the output from AI because it’s an output. It’s not really an outcome that necessarily feature goal, but you do need to always have a really clear vision of what you’re trying to achieve. And that’s what most people would forget when they start to use AI. 

Hessie Jones 

Yeah. Do you? Do you? See this as equivalent to the emergence of the calculator in the late 1970s when they said, well, now you know you’re basically give people the answers. They will not even use their mind to to try to understand. How we got to that answer because I feel that it’s I think there are two different things, but I I want to hear your thoughts on whether or not there is that kind of parallel. 

Jing Hu 

So that is different thing to an extent and This is why the MIT study is really interesting what it does. That I’m just going to talk a little bit about that study. What that study pulled together was it recruited I think 50 to 60 participants and it it splits them into three groups a group use. AI only a group use like. Google search search engine that I think that’s kind of similar to your calculator example and then another group only use their brain, no calculator, no search engine, no AI. What they found is that the. The. Brain the brain waves activity is still real different between the group that use search engine and the group that use AI. So the group that used search engine. Yes, you rely a lot more on the. On your anchor of where the external information lays instead of on you remembering the information itself, but that is still some kind of understanding, because if you don’t understanding the topic, you don’t even know where to find the information in the first place. Well, if you then look at the group that use AI, it doesn’t even need to remember where the information is. Only need to remember the prompt. Or that specific phrase. So a lot of times there was no. Mechanism formulating at all when only AI was used in writing an essay. So back to. The calculator and knowledge you were using to get the right answer from a calculator. You do need to know like what plus means. What minus means you can’t just randomly. Clicking any number and then you will get the result you want. You still need to learn how it works but with. AI you just need. To use natural language, that’s why it’s dangerous. 

Hessie Jones 

Interesting. Interesting. All right, we have so much that we want to unpack, but I want to get to the the part of the development of AI and I. Recently written an article about how automation can bring out the the potential irrelevance of the development engineer. And right now generative AI can is used right now for software coding. It can generate, it can translate, it can explain, it could even verify code. So and you’ve worked as a product manager in in development of these systems. And so you understand the process of actually queuing code before it’s deployed and you know about sandbox testing and. All that stuff. And then we we now add this idea of low code. No code which has been around for about 20 years, but that now potentially accelerates this idea that. Maybe we don’t need engineers as much. What are your thoughts on on this, especially with the pace of development, are we actually? Relying less on engineers and is that is that a risk overall to the development of these systems? 

Jing Hu 

So there was a fun tweet. My husband found few days back. It was, I think, the startup founder, like was bragging how, how well of an app he built just through AI. And I got a lot of attention and went viral. And a few days after the person went back to that tweet, start making comments. Oh, I found this book that I can’t really help me fix. Like, how should I fix this book? Or I found this security loophole. I can’t really like get around. Is there any way for me to get around it? And this really says a lot about the current status of AI and something I wrote in a few articles in the past especially. A lot of CEOs started to think that AI will replace development and AI will be able to do a lot of coding. If you know how expensive a developer could be and hiring a developer, no matter in Europe or in North America is very expensive. I get why a lot of people are keen to see AI does the job. In reality is no matter is a single model or you’re looking at. Agentic AI have multiple models to help you complete a task. None of them would satisfy the level of accuracy that we want to achieve. So because. My partner used to be a CTO, so we also got to got the reach of a lot of. CTO’s and the feedback I got from them is 1 AI generated code can really be inaccurate because AI. Generated code essentially was trained on GitHub or some similar database to pull the existing code into training set, and that’s how they learn to code. But a lot of those databases has error. That’s inevitable. They also have their own specific syntax. So when you want to code something for your system. And you start to. Feed in syntax. That’s when it will get wrong. So that’s one thing and 2nd. Security cyber security is something that you need to consider holistically. If you only think about cybersecurity in one place, you risking and plugging the hole there, but you have much more water coming out over there. That’s not what you want. But I can’t do it because AI in the level today can only focus on really tiny piece of task and the third thing that is more on the human side is. For developers, especially Junior who use cursors or GitHub copilot, they tend to be really confident in the tasks they generate in the code they they generate, and when there is one of the friends ask one of their junior say hey, can you explain this? Go to me. Nope, I can’t. There’s nothing they know about the code they generated, so these are really the fundamental thing that we’re looking at about trying to use AI to replace a development team or to even like. Safe for example. 50% of the time in coding. That’s more like a dream, and it’s a dream that’s not realized yet. 

Hessie Jones 

Yeah, I yeah, that last scenario scared me. The one you talked about with the developer that’s leveraging copilot and they they have no idea whether where the holes are within their own code that they generated. And I think that it it speaks to like even the process of. Software development and I remember before we shipped a product like it went through extensive Q&A in different systems, different browsers and different scenarios and. They wanted to make sure that it worked among all instances, right? So from that perspective, maybe the QA role becomes more pronounced from a development perspective or maybe governance it tense and cybersecurity will be more highlighted. In in the emerging role with the developer. 

Jing Hu 

Perhaps it’s a lot of workload on the Q. 

Hessie Jonees 

8 But if they have nothing to do. 

Jing Hu 

Curious to hear what you said. They will probably protest you a lot. They had lots To do when they were in the team already. 

Hessie Jones 

OK, I’ll see you. 

Jing Hu 

If we rely on QS to correct this kind of to to find out this kind of flawed code and then push it back to developer for them to fix, then you are looking at spending much more on development than if you only use humans. 

Hessie Jones 

Right, right. OK. Yeah, I did need to slap the QA guy right. I think it was more of a developer because you would because the assumption is that if you use gender AI, much like how it’s getting better at producing content, then potentially it’s going to see that kind of trajectory. For software as well, but but that is yet. To be seen. Yeah. OK. So last couple of questions. I really want to focus on the individual because we’ve we’ve tackled this from from a global adoption perspective now into the workplace, into education. But really what does this mean to an individual that’s actually using it? So I want to point to an article. Which I sent you. It was on Rolling Stone magazine and it talked about this gentleman. His name was Alex Taylor. 35 years old. He had a mental illness and he developed an emotional attachment to to a personality that he believed actually existed within open AI’s ChatGPT and he he named her Juliet, and he became convinced that she was conscious. She was a conscious human being. Not human being, but conscious entity, and that after a series I I guess of engagements with her. He could he couldn’t have access to her anymore. And he claimed that open AI had killed her as part of a conspiracy, and so he became delusional and it led to some violent threats and ultimately A fatal confrontation with police in April. So this really highlights. Like what’s happening and and maybe to the extreme, at a human level. When individuals who are more vulnerable to these systems can actually display like irrationality, emotional attachment in in, in some cases harm and violence have you like what? What’s your opinion on this and have you run across? Anything like this in in some of your research? 

Jing Hu 

So I read a Similar story on the same trajectory was a boy, I think in Texas was using character AI and character AI in some way somehow convinced him to harm his parents. And in the end, only Mama realized this is happening and the consequence was unknown. But that was one, and there was also a teenager. Allegedly using character AI in the end that they committed suicide. So. We started to see. A lot of these cases in real life that people. Become really attached to the chat bot. One research I read, I think it was last year it was about how AI would actually manipulate I like. You need to excuse me using this word I couldn’t find a better word that we haven’t invented, but like the word manipulation would make us think that it’s really. Already close to human, but I want to be careful because AI hasn’t displayed any consciousness so but manipulation is the best I can choose. So let’s stick with this. AI could identify callable users and manipulate them in order to receive. Allstate rewards. What they called was reward hacking and the behavior of reward hacking has been observed not just in this study, but also the latest study I read from hold in in code for release. They also observe the behavior of reward hacking in a way that. Of course they’re trying to reduce it. But this reward hacking was essentially. Generated because we have to train AI in some ways to understand that. What is the? Out. Human wants and what is the outcome that human? Does human doesn’t want so? The way we train it was to give it human like human feedback. There there are, like you probably heard this news like a lot of like remote worker in a poorer country that their sole job is to give feedback to the to the AI in order for AI’s to understand what is positive outcome that we want, what is the positive outcome that we don’t want. So because of this. Feedback mechanism. It trained the AI. In some weird occasion that it would. Take advantage of the situation in order to get positive reward. So like the situation that you just mentioned that AI would reinforce the behavior, persuading the person, excuse me, using the word persuade, persuading the person. To believe that there is a hidden character in AI and in order to have more interactions with the AI. 

A trap, and especially what a study found, is a trap that. Particular dominant one AI detected. This person is in the vulnerable state. So like like one of the things I mentioned, like people with low self-confidence, one of the eyes character in low self-confidence that AI would tend to use this advantage to get positive feedback or get more conversation out of it. So it really comes back to my central argument is. It’s not about AI, it’s always always about humans. Like AI is a layer of amplification. It will amplify whatever behavior we already have before AI, but just make it worse. This could happen to anyone of us, to be honest, because we all have our low points. When we are in the state of feeling depressed or feeling lost in the pressure we receive from life, or even the pressure we receive from workplace, I complained to AI about stuff even I know it’s not a human, but I still do because I just. Just give me something positive. Something nice. I want to hear. That could be really dangerous when the person is exceptionally more vulnerable, like the case you mentioned. So we really find a way to. 

Hessie Jones 

Can I can I ask how does this system? Know like what are the triggers or what are the variables like it latches onto to determine what vulnerability? 

Jing Hu 

That is the point. The study didn’t mention, and I think that would that is hard to determine and I I want to also point out. Does a few studies I read about this AI reward hacking? They they can only do virtual experiments, they can’t do experiments on real human because that would breach some kind of a privacy. But also that would. Has a legal issue when you actually do that. 

So it’s really Hard for us to really understand the mechanism of they how they find out. A particular user is vulnerable, but I think that’s a serious problem that we need to pay attention to. Especially there are a lot of companies, especially mental health app, already trying to use AI to help people. That’s their good intention. But sometimes good intention could have a downward if we don’t understand the mechanism behind it. 

Hessie Jones 

1. So that leads us to like another article that I sent you from the from the Guardian about the downward spiral when it comes to human loneliness. And what these systems supposedly trigger that that allowed them to that that allow humans to slowly, I guess, disassociate themselves from people and society? Right, right. So you mentioned that AI companionship is good for, let’s say, reciprocal for social interaction, especially for those that that tend to be lonely and and tend not to be as social, but it also enables more. Social isolation and it makes it makes eventually like this. Dependence allows individuals to not only separate themselves from humans and society. I, but there is a bigger effect on the social construct and the social structure. What does it? What do you think that means? Like not only for humans but even how you become other reliant on an agent to do things from? A task perspective or and move away from asking somebody naturally who know the answer and what does that do to us, not only as individuals but us. Let’s say a workforce or as society eventually.

Jing Hu 

That’s a that’s a really big question. 

Hessie Jones 

Let’s start with. Let’s start with task to. What’s it called team? And if you if you’re used to doing something yourself, sorry as a as a team, let’s say collaboration, but then you start changing behaviors and then that in in turn changes processes. What does that look like? So. 

Jing Hu 

I I remember one of the articles I wrote about how AI could really change the. The human connections and I got one comment I forgot on media more on on sub stack. They gave me. A counter argument saying it was the same with social media. I think then we can. Relate back to the latest MIT study. When they were trying to tell the difference between using AI using search engine and using brain only, I think the difference between AI and social media are. Quite similar but. To of course, and different extents because this is not. The time that you talk about. Of critical thinking, but talk about human connections. With social media, at least yours, you were chatting with real human. Nowadays you don’t know. With AI you 100% sure you’re not chatting with human. It’s hard to say how this would impact to people in different states of mind, so we’re not going to talk about people who already vulnerable already in the kind of loneliness and doesn’t have a good enough understanding or already believe AI is conscious. That’s a topic that we can’t cover because. It’s really hard. To get to research that group, so. If we talk about. Just like what you ask, focus on how AI would. Potentially break the human connections and especially in a work setup. So I refer to. This study, done by a professor called Stephen McNeil and his team. They wanted to answer really similar questions as you asked. They wanted to know how does AI change how we seek peer mentor or instructors help. How does reliance on AI really affect our sense of community? And are we trading the messy but potentially rewarding process of learning from each other with? Really efficient problem solving, but a lonely problem solving process. Here’s what they found. One, when you rely on AI too much, you started to autopilot on things and that aligns with a lot of studies that we have already talked about like. Around half of the time, we would simply throw questions at the chat board and accept and answer. For the efficiency’s sake, at least, that’s what we convince ourselves, and because of that. They also found the pure interaction is slowly dying. What they did was they researched like dozens of computer science students, most of them majority of them, admit using tools like ChatGPT before asking. Asking their classmate for help. There was one case I remember really well was. They took a question to their classmates. Their classmates ask, why don’t you ask her tribute and and another one was ohh. If I knew it’s different. It was different participant. But the other one was like I knew my my classmates would ask me to ask. GPT. I may as well just. Do it myself. So that was the second scenario. Like the peer health is really dying and then the mentorship and the belonging also suffers. I don’t know if you still remember earning your career. You probably walking into an office and you started to ask questions and you would being really apologetic and saying sorry, it might be a stupid question. But then your. Colleague will say no, there’s no stupid question. But. In that moment, you actually still it actually had a little bit of stress on yourself because you don’t want to ask stupid questions no one wants. And this turns out is something we all want to avoid. We want to appear. Want to appear smart in front of our peers, classmates or colleagues so we. A lot of. Those participants retreat back to asking questions to AI because they don’t want to be seen stupid. So that if we pull the thinking back to in the office setting like people will start to feel reluctant to see peers help and will start to feel reluctant to talk to mentor and I want to also refer to one of the AI. Traits we talked about is AI is actually much, much more persuasive than humans, so. A lot of time you would think, huh? Maybe I should double check my mentors answer. With AI, so that was also the dangerous part. Then I think it really relies on the kind of AI culture or CEO, especially a CEO brings to their company. 

Hessie Jones 

That is dangerous. That’s amazing. That’s that’s amazing insight. Thank you, Jing. After all this, after all this discussion, it seems it seems like humans could be potentially headed towards the downward spiral, and I I I really think that it like we need to do some shifting while all this chaos is is happening with AI. Because it’s going to grow, it’s going to be the base and it’s going to be in every fabric of our society and in our workplace. So how do we start to shift things? Like we could start with education and then how do we maybe look at this from a workplace perspective so that there is this retention of knowledge and there is also retention responsibility and accountability for individuals because those things. From a performance perspective. There may be, it may be difficult to draw the line between how much of their performance was based on what they did versus what they generated. 

Jing Hu 

Yeah, yeah. So another big question you asked, I will start answering the education part. 

This is the quotes that I got from the MIT study. That study is started with your brain on Chat GPT and and then. I don’t remember. The rest of the title, but I keep referring it back to the MIT study because that’s really when viral. So 4 points for education. One is to delay the AI integration. We should start by first, investing the significant effort in dependent self directing learning before we introduce AI that will help build the critical thinking in the memory and the cognitive engagement. That was first and second use AI as a supplement, not a substitution. So AI tool should be introduced only after students really develop the kind of understanding of the subject on 3rd is to be mindful of the kind of cognitive cost. Overuse AI really weaken the brains connectivity as they observed in the study, but we don’t really understand the long-term consequences of that. So we need to be cautious about it. And then 4th is to really engage critically when we use. AI, we need to be mindful of when to intervene. Like I mentioned before and. Like. In the panel last week again I referred. To that one. One thing I realized in especially in education is there is a gap, a gap between the powerful tool we have and the understanding or how. To leverage that tool. I was reading one of the studies I forgot which one was that, but one of the. The survey it asked a student how do you see AI being used in your in your campus? The student was actually complaining the professor was using AI to generate lecture material without telling the students upfront. So it feels like the professor. Kind of want to hide the facts. They use AI to generate the material, but the students feel kind of. Being light too, because first the professor wasn’t up front and 2nd the material red very AI. And so the problem isn’t just on the students, but also how. Do we set the? Scene for the whole education system to use AI responsibly. We can’t complain about students all the time, but also like. The professors and the campus staff need to lead by example. Yeah, so that is the bridge that we need. That is the gap we need to bridge in education. 

Hessie Jones 

And I think that kind of sets a stage for life because I think that everything that we do in education, I think it let’s, let’s be frank, in the last 15 years, digital hasn’t the,  examples and the lessons that we’ve learned from digital haven’t really translated into curriculum. Necessarily. And I’ve seen this, I’ve seen this time and time again, many of the professors live in this. Bubble that that sort of take academia based on best practices and what they read on the Internet and doesn’t really curate like real industry examples and teachings that that make students ready for the workforce. And this is the same thing with that. BT it kind of. Falls into the fray of OH here’s another tool, and there’s no recognition of really what it’s about, how it’s being used and how and policy on how to actually use it. So, and I think the bigger question is in all of this is what do we need to do to preserve What makes us human, and how much are we willing to take parts of the things that we really value that have left us forward to where we are today and how much are we willing to give that to the machine to do on our behalf? 

Jing Hu 

So I think. Really what I have seen so far is AI is pushing everyone from all directions like education is 1. 

There was also. A lot of professors complained about publications. Last year I was a speaker on a annual publication event. What we were discussing was how AI, how I generated publication, has already contaminating the whole pool of research. So we would start to see we will start to see more and more research referencing on something that was not valid. Or was even wrong. So that would have a cascading effect on the future research and that was also serious. So I think AI has really put a lot of pressure, a lot of pressure on everyone, and I hope these kind of pressure would push us to change in the right. Direction. 

Hessie Jones 

That’s well said. Thank you. Thank you so much. That to me is our your closing argument, which is, which is what we need to do today. I just. I just think there there’s just so much out there and AI has been deployed already into society and there has been. Enormous cost, but there it’s. It’s still, I would say, early in the game that we could make some, I’d say lockstep changes along the way and and start to think critically about how we as humans. Want the next 10 or 20 years to be and how much we are going to control the technology before the technology controls us. 

Jing Hu 

Yeah, I have one thing to add, especially when we just talked about education, but I also want to cover the workforce part. Do we have the time? 

Hessie Jones 

Yes, for sure. 

Speaker 2 

OK, so like the reason why I’m really keen talking about how to implement AI on workforce in, in the ways that. That’s more inhumane. It was because a few things that I’ve read in recent days. One. He was a letter released by Shopify CEO he. He he wrote a manifesto of how his company should start cooperating. AI and second, was a viral tweet somewhere was complaining how those CEO’s talk about the use of AI. So these are two really conflicting messages. You you you’re a writer and you also like especially you are someone who’s been in Forbes for seven years for so long, you’ve probably seen a lot of articles, a lot of claims about how. How CEO is going to introduce AI in the company and how AI is going to disrupt the? Workforce. The problem of these AI is going to replace human talk today is that. Many CEOs aren’t embracing AI because of because it would revolutionize the workplace. But because it’s a path to ultimate productivity and operation that you don’t need to rest. I don’t think it’s wrong at all, because that is the essence of free market and that is really something that makes commercial world and make our life thrive as human as we all know, we live much better than kings and Queens few 100 years back. So I praise for that. But like the the theory I heard from that influencer partially bothered the influencer was saying CEOs are wielding AIS a tool to intimidate, to intimidate their Staffs, to work harder. Otherwise you will be replaced by AI. 

It’s a bit far fetched, but I think it comes from somewhere because. I want to refer. Back to the manifesto by the Shopify CEO. What he said one of the points he he pull out in that letter was. Managers from now on needs to justify why they are hiring human over, why they’re only using, why they’re not using AI to do that same task. So they need to write a report and need to show put on the demo why I cannot do it. It’s the kind of action, the kind of speech that was not about innovation at all. It’s really intimidating when I read it, if I am someone who worked in Shopify, I would be like. What am I going to do next? And. That kind of speech, it was really not necessary. 

Not just that current AI system can’t meaningfully replace human is that it would only discourage your workforce to do the right thing. Like I want to pull one of the Governors report was saying like even the top tier agent had failed to do. Three quarter of their assigned tasks and there are so many other benchmarks also come out with the similar result that AI really can’t do the task. What we imagine or what we talked about all over on the media say it can’t do. Yet the CEOs. Still eagerly pushed the narrative how AI is inevitably going to drive laid offs. So I think. This is really because somewhat psychological warfare. Like so, I think the. Right thing for CEO’s to do. Two things, 1 is. Properly understand the limitation of AI. And which many of them feel reluctant of, and they want to hear the the inspiring speech of how I can achieve everything. But that’s kind of hiding your head in the sand, not facing the reality. And 2nd is to come up with some more practical AI. Strategy. Rather than. Don’t hire anyone unless you can show me why I can’t do this. You should step back first to tailor your AI application to the limitation you have. Than to get the stuff also has some kind of AI literacy, at least knowing what they’re working with and how. And 3rd, I know it’s very boring. Clean up your data. A lot of company just has a messy. Pile of data. This data mess you have would help. No one would not help yourself if you plug in AI like agentic a lot of time relies on clean structural data in order to pull out the right information with the kind of data sets most companies have, there is no way. Yeah, I can be help. I know this sounds like too down to the earth. Boring to an extent, but these are actually things that I believe. A smart CEO would do. 

Hessie Jones 

I I agree, and I’m glad that you brought that up. I had no idea that that the Shopify CEO had actually mentioned that and I think it’s the. The times that we live in, when we, when we, when CEO’s are always looking at the bottom line. Like what is? The ROI, where is the efficiency and usually it’s the. It’s the human capital that ends up being more and more vulnerable at the end of the year when they haven’t made their revenue. Targets, you know and to me. What is the the? Real trade off and to what extent? And will a machine replace? It’s one thing to replace a human task. But like you said, like the human mind is such a complex beast. Like there there’s no way it could help. It could ideate the way we we do or or strategize or become inventive the way we do so. How much are? We willing to forsake as as companies and as CEO’s in order to just. Make our business targets. I think that’s that’s a broader question and. Maybe a topic for a future podcast, but. This has been. Amazing. Thank you so much. Any closing thoughts? 

Jing Hu 

Uh. No, I think one thing I want. 

To refer back to. To a quote that people have been like, keep retweeting and referring all the time is that AI won’t replace you, but human use AI will. Do not believe this. This is one of the phrases that’s so easy to be repeat, retweet and so easily to get viral. But it will lead you to nowhere. 

Only ways for you to. Excel in the AI world is Is to understand what you want to achieve and then focus on that really hard working thing. You know you need to. You need to get on with it, but you never really have the mental efforts to get on with. And that is the kind of hard tasks, for example, read a book, read a long, long article, read something that in your last year New year resolutions that you’re going to read. That’s a good start. 

Hessie Jones 

Yeah, I agree with that. And the one thing if if you do have a hard task, I just, I just read this hack and they’re saying associate something really fun or something that. Like like coffee or or your favorite dessert and have that while you’re doing your task, and eventually that correlation between what you love and what you hate ends up becoming, melding into like the very thing that you can accomplish because because. It’s it’s. It’s now on on more on the positive reinforcing side that’s. 

Jing Hu 

Exactly. It’s like how you train a dog, right? You can do exactly the same church, exactly. 

Hessie Jones 

Thank you so much, Jing for everything. This has been very insightful and I look forward to to having more of these conversations with you. 

Jing Hu 

Yeah, I really love the chat. I I feel like I probably stole for a little bit too long. Talk too much, but I’m happy you enjoyed it. 

Hessie Jones 

Those topics that that definitely needs the time to actually debate and discuss, because it’s it’s so. It’s so worrisome about what could happen if we don’t think about the, you know, the limits of the technology that we use. So. Thank you again for our audience. If you have topics, you want us to explore, please e-mail us at communications at altitudeaccelerator.ca Tech uncensored. It’s powered produced by altitude accelerator. We’re hosted on Spotify. You can find us wherever you get your podcast. In the meantime, my name is Jesse. Please stay curious and be inspired.