definition of complex problem solving

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

What It Takes to Think Deeply About Complex Problems

• Tony Schwartz

Three ways to embrace a more nuanced, spacious perspective.

The problems we’re facing often seem as intractable as they do complex. But as Albert Einstein famously observed, “We cannot solve our problems with the same level of thinking that created them.” So what does it take to increase the complexity of our thinking? To cultivate a more nuanced, spacious perspective, start by challenging your convictions. Ask yourself, “What am I not seeing here?” and “What else might be true?” Second, do your most challenging task first every day, when your mind is fresh and before distractions arise. And third, pay attention to how you’re feeling. Embracing complexity means learning to better manage tough emotions like fear and anger.

The problems we’re facing often seem as complex as they do intractable. And as Albert Einstein is often quoted as saying, “We cannot solve our problems with the same level of thinking that created them.” So what does it take to increase the complexity of our thinking?

• Tony Schwartz is the CEO of The Energy Project and the author of The Way We’re Working Isn’t Working . Become a fan of The Energy Project on Facebook .

Partner Center

35 problem-solving techniques and methods for solving complex problems

Design your next session with SessionLab

Join the 150,000+ facilitators 
using SessionLab.

Recommended Articles

A step-by-step guide to planning a workshop, how to create an unforgettable training session in 8 simple steps, 47 useful online tools for workshop planning and meeting facilitation.

All teams and organizations encounter challenges as they grow. There are problems that might occur for teams when it comes to miscommunication or resolving business-critical issues . You may face challenges around growth , design , user engagement, and even team culture and happiness. In short, problem-solving techniques should be part of every team’s skillset.

Problem-solving methods are primarily designed to help a group or team through a process of first identifying problems and challenges , ideating possible solutions , and then evaluating the most suitable .

Finding effective solutions to complex problems isn’t easy, but by using the right process and techniques, you can help your team be more efficient in the process.

So how do you develop strategies that are engaging, and empower your team to solve problems effectively?

In this blog post, we share a series of problem-solving tools you can use in your next workshop or team meeting. You’ll also find some tips for facilitating the process and how to enable others to solve complex problems.

Let’s get started! 

How do you identify problems?

How do you identify the right solution.

• Tips for more effective problem-solving

Complete problem-solving methods

• Problem-solving techniques to identify and analyze problems
• Problem-solving techniques for developing solutions

Problem-solving warm-up activities

Closing activities for a problem-solving process.

Before you can move towards finding the right solution for a given problem, you first need to identify and define the problem you wish to solve. 

Here, you want to clearly articulate what the problem is and allow your group to do the same. Remember that everyone in a group is likely to have differing perspectives and alignment is necessary in order to help the group move forward. 

Identifying a problem accurately also requires that all members of a group are able to contribute their views in an open and safe manner. It can be scary for people to stand up and contribute, especially if the problems or challenges are emotive or personal in nature. Be sure to try and create a psychologically safe space for these kinds of discussions.

Remember that problem analysis and further discussion are also important. Not taking the time to fully analyze and discuss a challenge can result in the development of solutions that are not fit for purpose or do not address the underlying issue.

Successfully identifying and then analyzing a problem means facilitating a group through activities designed to help them clearly and honestly articulate their thoughts and produce usable insight.

With this data, you might then produce a problem statement that clearly describes the problem you wish to be addressed and also state the goal of any process you undertake to tackle this issue.  

Finding solutions is the end goal of any process. Complex organizational challenges can only be solved with an appropriate solution but discovering them requires using the right problem-solving tool.

After you’ve explored a problem and discussed ideas, you need to help a team discuss and choose the right solution. Consensus tools and methods such as those below help a group explore possible solutions before then voting for the best. They’re a great way to tap into the collective intelligence of the group for great results!

Remember that the process is often iterative. Great problem solvers often roadtest a viable solution in a measured way to see what works too. While you might not get the right solution on your first try, the methods below help teams land on the most likely to succeed solution while also holding space for improvement.

Every effective problem solving process begins with an agenda . A well-structured workshop is one of the best methods for successfully guiding a group from exploring a problem to implementing a solution.

In SessionLab, it’s easy to go from an idea to a complete agenda . Start by dragging and dropping your core problem solving activities into place . Add timings, breaks and necessary materials before sharing your agenda with your colleagues.

The resulting agenda will be your guide to an effective and productive problem solving session that will also help you stay organized on the day!

Tips for more effective problem solving

Problem-solving activities are only one part of the puzzle. While a great method can help unlock your team’s ability to solve problems, without a thoughtful approach and strong facilitation the solutions may not be fit for purpose.

Let’s take a look at some problem-solving tips you can apply to any process to help it be a success!

Clearly define the problem

Jumping straight to solutions can be tempting, though without first clearly articulating a problem, the solution might not be the right one. Many of the problem-solving activities below include sections where the problem is explored and clearly defined before moving on.

This is a vital part of the problem-solving process and taking the time to fully define an issue can save time and effort later. A clear definition helps identify irrelevant information and it also ensures that your team sets off on the right track.

Don’t jump to conclusions

It’s easy for groups to exhibit cognitive bias or have preconceived ideas about both problems and potential solutions. Be sure to back up any problem statements or potential solutions with facts, research, and adequate forethought.

The best techniques ask participants to be methodical and challenge preconceived notions. Make sure you give the group enough time and space to collect relevant information and consider the problem in a new way. By approaching the process with a clear, rational mindset, you’ll often find that better solutions are more forthcoming.  

Try different approaches  

Problems come in all shapes and sizes and so too should the methods you use to solve them. If you find that one approach isn’t yielding results and your team isn’t finding different solutions, try mixing it up. You’ll be surprised at how using a new creative activity can unblock your team and generate great solutions.

Don’t take it personally 

Depending on the nature of your team or organizational problems, it’s easy for conversations to get heated. While it’s good for participants to be engaged in the discussions, ensure that emotions don’t run too high and that blame isn’t thrown around while finding solutions.

You’re all in it together, and even if your team or area is seeing problems, that isn’t necessarily a disparagement of you personally. Using facilitation skills to manage group dynamics is one effective method of helping conversations be more constructive.

Get the right people in the room

Your problem-solving method is often only as effective as the group using it. Getting the right people on the job and managing the number of people present is important too!

If the group is too small, you may not get enough different perspectives to effectively solve a problem. If the group is too large, you can go round and round during the ideation stages.

Creating the right group makeup is also important in ensuring you have the necessary expertise and skillset to both identify and follow up on potential solutions. Carefully consider who to include at each stage to help ensure your problem-solving method is followed and positioned for success.

Document everything

The best solutions can take refinement, iteration, and reflection to come out. Get into a habit of documenting your process in order to keep all the learnings from the session and to allow ideas to mature and develop. Many of the methods below involve the creation of documents or shared resources. Be sure to keep and share these so everyone can benefit from the work done!

Bring a facilitator 

Facilitation is all about making group processes easier. With a subject as potentially emotive and important as problem-solving, having an impartial third party in the form of a facilitator can make all the difference in finding great solutions and keeping the process moving. Consider bringing a facilitator to your problem-solving session to get better results and generate meaningful solutions!

Develop your problem-solving skills

It takes time and practice to be an effective problem solver. While some roles or participants might more naturally gravitate towards problem-solving, it can take development and planning to help everyone create better solutions.

You might develop a training program, run a problem-solving workshop or simply ask your team to practice using the techniques below. Check out our post on problem-solving skills to see how you and your group can develop the right mental process and be more resilient to issues too!

Design a great agenda

Workshops are a great format for solving problems. With the right approach, you can focus a group and help them find the solutions to their own problems. But designing a process can be time-consuming and finding the right activities can be difficult.

Check out our workshop planning guide to level-up your agenda design and start running more effective workshops. Need inspiration? Check out templates designed by expert facilitators to help you kickstart your process!

In this section, we’ll look at in-depth problem-solving methods that provide a complete end-to-end process for developing effective solutions. These will help guide your team from the discovery and definition of a problem through to delivering the right solution.

If you’re looking for an all-encompassing method or problem-solving model, these processes are a great place to start. They’ll ask your team to challenge preconceived ideas and adopt a mindset for solving problems more effectively.

• Six Thinking Hats
• Lightning Decision Jam
• Problem Definition Process
• Discovery & Action Dialogue

Design Sprint 2.0

• Open Space Technology

1. Six Thinking Hats

Individual approaches to solving a problem can be very different based on what team or role an individual holds. It can be easy for existing biases or perspectives to find their way into the mix, or for internal politics to direct a conversation.

Six Thinking Hats is a classic method for identifying the problems that need to be solved and enables your team to consider them from different angles, whether that is by focusing on facts and data, creative solutions, or by considering why a particular solution might not work.

Like all problem-solving frameworks, Six Thinking Hats is effective at helping teams remove roadblocks from a conversation or discussion and come to terms with all the aspects necessary to solve complex problems.

2. Lightning Decision Jam

Featured courtesy of Jonathan Courtney of AJ&Smart Berlin, Lightning Decision Jam is one of those strategies that should be in every facilitation toolbox. Exploring problems and finding solutions is often creative in nature, though as with any creative process, there is the potential to lose focus and get lost.

Unstructured discussions might get you there in the end, but it’s much more effective to use a method that creates a clear process and team focus.

In Lightning Decision Jam, participants are invited to begin by writing challenges, concerns, or mistakes on post-its without discussing them before then being invited by the moderator to present them to the group.

From there, the team vote on which problems to solve and are guided through steps that will allow them to reframe those problems, create solutions and then decide what to execute on. 

By deciding the problems that need to be solved as a team before moving on, this group process is great for ensuring the whole team is aligned and can take ownership over the next stages. 

Lightning Decision Jam (LDJ)   #action   #decision making   #problem solving   #issue analysis   #innovation   #design   #remote-friendly   The problem with anything that requires creative thinking is that it’s easy to get lost—lose focus and fall into the trap of having useless, open-ended, unstructured discussions. Here’s the most effective solution I’ve found: Replace all open, unstructured discussion with a clear process. What to use this exercise for: Anything which requires a group of people to make decisions, solve problems or discuss challenges. It’s always good to frame an LDJ session with a broad topic, here are some examples: The conversion flow of our checkout Our internal design process How we organise events Keeping up with our competition Improving sales flow

3. Problem Definition Process

While problems can be complex, the problem-solving methods you use to identify and solve those problems can often be simple in design. 

By taking the time to truly identify and define a problem before asking the group to reframe the challenge as an opportunity, this method is a great way to enable change.

Begin by identifying a focus question and exploring the ways in which it manifests before splitting into five teams who will each consider the problem using a different method: escape, reversal, exaggeration, distortion or wishful. Teams develop a problem objective and create ideas in line with their method before then feeding them back to the group.

This method is great for enabling in-depth discussions while also creating space for finding creative solutions too!

Problem Definition   #problem solving   #idea generation   #creativity   #online   #remote-friendly   A problem solving technique to define a problem, challenge or opportunity and to generate ideas.

4. The 5 Whys 

Sometimes, a group needs to go further with their strategies and analyze the root cause at the heart of organizational issues. An RCA or root cause analysis is the process of identifying what is at the heart of business problems or recurring challenges. 

The 5 Whys is a simple and effective method of helping a group go find the root cause of any problem or challenge and conduct analysis that will deliver results. 

By beginning with the creation of a problem statement and going through five stages to refine it, The 5 Whys provides everything you need to truly discover the cause of an issue.

The 5 Whys   #hyperisland   #innovation   This simple and powerful method is useful for getting to the core of a problem or challenge. As the title suggests, the group defines a problems, then asks the question “why” five times, often using the resulting explanation as a starting point for creative problem solving.

5. World Cafe

World Cafe is a simple but powerful facilitation technique to help bigger groups to focus their energy and attention on solving complex problems.

World Cafe enables this approach by creating a relaxed atmosphere where participants are able to self-organize and explore topics relevant and important to them which are themed around a central problem-solving purpose. Create the right atmosphere by modeling your space after a cafe and after guiding the group through the method, let them take the lead!

Making problem-solving a part of your organization’s culture in the long term can be a difficult undertaking. More approachable formats like World Cafe can be especially effective in bringing people unfamiliar with workshops into the fold. 

World Cafe   #hyperisland   #innovation   #issue analysis   World Café is a simple yet powerful method, originated by Juanita Brown, for enabling meaningful conversations driven completely by participants and the topics that are relevant and important to them. Facilitators create a cafe-style space and provide simple guidelines. Participants then self-organize and explore a set of relevant topics or questions for conversation.

6. Discovery & Action Dialogue (DAD)

One of the best approaches is to create a safe space for a group to share and discover practices and behaviors that can help them find their own solutions.

With DAD, you can help a group choose which problems they wish to solve and which approaches they will take to do so. It’s great at helping remove resistance to change and can help get buy-in at every level too!

This process of enabling frontline ownership is great in ensuring follow-through and is one of the methods you will want in your toolbox as a facilitator.

Discovery & Action Dialogue (DAD)   #idea generation   #liberating structures   #action   #issue analysis   #remote-friendly   DADs make it easy for a group or community to discover practices and behaviors that enable some individuals (without access to special resources and facing the same constraints) to find better solutions than their peers to common problems. These are called positive deviant (PD) behaviors and practices. DADs make it possible for people in the group, unit, or community to discover by themselves these PD practices. DADs also create favorable conditions for stimulating participants’ creativity in spaces where they can feel safe to invent new and more effective practices. Resistance to change evaporates as participants are unleashed to choose freely which practices they will adopt or try and which problems they will tackle. DADs make it possible to achieve frontline ownership of solutions.

7. Design Sprint 2.0

Want to see how a team can solve big problems and move forward with prototyping and testing solutions in a few days? The Design Sprint 2.0 template from Jake Knapp, author of Sprint, is a complete agenda for a with proven results.

Developing the right agenda can involve difficult but necessary planning. Ensuring all the correct steps are followed can also be stressful or time-consuming depending on your level of experience.

Use this complete 4-day workshop template if you are finding there is no obvious solution to your challenge and want to focus your team around a specific problem that might require a shortcut to launching a minimum viable product or waiting for the organization-wide implementation of a solution.

8. Open space technology

Open space technology- developed by Harrison Owen – creates a space where large groups are invited to take ownership of their problem solving and lead individual sessions. Open space technology is a great format when you have a great deal of expertise and insight in the room and want to allow for different takes and approaches on a particular theme or problem you need to be solved.

Start by bringing your participants together to align around a central theme and focus their efforts. Explain the ground rules to help guide the problem-solving process and then invite members to identify any issue connecting to the central theme that they are interested in and are prepared to take responsibility for.

Once participants have decided on their approach to the core theme, they write their issue on a piece of paper, announce it to the group, pick a session time and place, and post the paper on the wall. As the wall fills up with sessions, the group is then invited to join the sessions that interest them the most and which they can contribute to, then you’re ready to begin!

Everyone joins the problem-solving group they’ve signed up to, record the discussion and if appropriate, findings can then be shared with the rest of the group afterward.

Open Space Technology   #action plan   #idea generation   #problem solving   #issue analysis   #large group   #online   #remote-friendly   Open Space is a methodology for large groups to create their agenda discerning important topics for discussion, suitable for conferences, community gatherings and whole system facilitation

Techniques to identify and analyze problems

Using a problem-solving method to help a team identify and analyze a problem can be a quick and effective addition to any workshop or meeting.

While further actions are always necessary, you can generate momentum and alignment easily, and these activities are a great place to get started.

We’ve put together this list of techniques to help you and your team with problem identification, analysis, and discussion that sets the foundation for developing effective solutions.

Let’s take a look!

• The Creativity Dice
• Fishbone Analysis
• Problem Tree
• SWOT Analysis
• Agreement-Certainty Matrix
• The Journalistic Six
• LEGO Challenge
• What, So What, Now What?
• Journalists

Individual and group perspectives are incredibly important, but what happens if people are set in their minds and need a change of perspective in order to approach a problem more effectively?

Flip It is a method we love because it is both simple to understand and run, and allows groups to understand how their perspectives and biases are formed. 

Participants in Flip It are first invited to consider concerns, issues, or problems from a perspective of fear and write them on a flip chart. Then, the group is asked to consider those same issues from a perspective of hope and flip their understanding.  

No problem and solution is free from existing bias and by changing perspectives with Flip It, you can then develop a problem solving model quickly and effectively.

Flip It!   #gamestorming   #problem solving   #action   Often, a change in a problem or situation comes simply from a change in our perspectives. Flip It! is a quick game designed to show players that perspectives are made, not born.

10. The Creativity Dice

One of the most useful problem solving skills you can teach your team is of approaching challenges with creativity, flexibility, and openness. Games like The Creativity Dice allow teams to overcome the potential hurdle of too much linear thinking and approach the process with a sense of fun and speed. 

In The Creativity Dice, participants are organized around a topic and roll a dice to determine what they will work on for a period of 3 minutes at a time. They might roll a 3 and work on investigating factual information on the chosen topic. They might roll a 1 and work on identifying the specific goals, standards, or criteria for the session.

Encouraging rapid work and iteration while asking participants to be flexible are great skills to cultivate. Having a stage for idea incubation in this game is also important. Moments of pause can help ensure the ideas that are put forward are the most suitable. 

The Creativity Dice   #creativity   #problem solving   #thiagi   #issue analysis   Too much linear thinking is hazardous to creative problem solving. To be creative, you should approach the problem (or the opportunity) from different points of view. You should leave a thought hanging in mid-air and move to another. This skipping around prevents premature closure and lets your brain incubate one line of thought while you consciously pursue another.

11. Fishbone Analysis

Organizational or team challenges are rarely simple, and it’s important to remember that one problem can be an indication of something that goes deeper and may require further consideration to be solved.

Fishbone Analysis helps groups to dig deeper and understand the origins of a problem. It’s a great example of a root cause analysis method that is simple for everyone on a team to get their head around. 

Participants in this activity are asked to annotate a diagram of a fish, first adding the problem or issue to be worked on at the head of a fish before then brainstorming the root causes of the problem and adding them as bones on the fish. 

Using abstractions such as a diagram of a fish can really help a team break out of their regular thinking and develop a creative approach.

Fishbone Analysis   #problem solving   ##root cause analysis   #decision making   #online facilitation   A process to help identify and understand the origins of problems, issues or observations.

12. Problem Tree 

Encouraging visual thinking can be an essential part of many strategies. By simply reframing and clarifying problems, a group can move towards developing a problem solving model that works for them. 

In Problem Tree, groups are asked to first brainstorm a list of problems – these can be design problems, team problems or larger business problems – and then organize them into a hierarchy. The hierarchy could be from most important to least important or abstract to practical, though the key thing with problem solving games that involve this aspect is that your group has some way of managing and sorting all the issues that are raised.

Once you have a list of problems that need to be solved and have organized them accordingly, you’re then well-positioned for the next problem solving steps.

Problem tree   #define intentions   #create   #design   #issue analysis   A problem tree is a tool to clarify the hierarchy of problems addressed by the team within a design project; it represents high level problems or related sublevel problems.

13. SWOT Analysis

Chances are you’ve heard of the SWOT Analysis before. This problem-solving method focuses on identifying strengths, weaknesses, opportunities, and threats is a tried and tested method for both individuals and teams.

Start by creating a desired end state or outcome and bare this in mind – any process solving model is made more effective by knowing what you are moving towards. Create a quadrant made up of the four categories of a SWOT analysis and ask participants to generate ideas based on each of those quadrants.

Once you have those ideas assembled in their quadrants, cluster them together based on their affinity with other ideas. These clusters are then used to facilitate group conversations and move things forward. 

SWOT analysis   #gamestorming   #problem solving   #action   #meeting facilitation   The SWOT Analysis is a long-standing technique of looking at what we have, with respect to the desired end state, as well as what we could improve on. It gives us an opportunity to gauge approaching opportunities and dangers, and assess the seriousness of the conditions that affect our future. When we understand those conditions, we can influence what comes next.

14. Agreement-Certainty Matrix

Not every problem-solving approach is right for every challenge, and deciding on the right method for the challenge at hand is a key part of being an effective team.

The Agreement Certainty matrix helps teams align on the nature of the challenges facing them. By sorting problems from simple to chaotic, your team can understand what methods are suitable for each problem and what they can do to ensure effective results. 

If you are already using Liberating Structures techniques as part of your problem-solving strategy, the Agreement-Certainty Matrix can be an invaluable addition to your process. We’ve found it particularly if you are having issues with recurring problems in your organization and want to go deeper in understanding the root cause. 

Agreement-Certainty Matrix   #issue analysis   #liberating structures   #problem solving   You can help individuals or groups avoid the frequent mistake of trying to solve a problem with methods that are not adapted to the nature of their challenge. The combination of two questions makes it possible to easily sort challenges into four categories: simple, complicated, complex , and chaotic .  A problem is simple when it can be solved reliably with practices that are easy to duplicate.  It is complicated when experts are required to devise a sophisticated solution that will yield the desired results predictably.  A problem is complex when there are several valid ways to proceed but outcomes are not predictable in detail.  Chaotic is when the context is too turbulent to identify a path forward.  A loose analogy may be used to describe these differences: simple is like following a recipe, complicated like sending a rocket to the moon, complex like raising a child, and chaotic is like the game “Pin the Tail on the Donkey.”  The Liberating Structures Matching Matrix in Chapter 5 can be used as the first step to clarify the nature of a challenge and avoid the mismatches between problems and solutions that are frequently at the root of chronic, recurring problems.

Organizing and charting a team’s progress can be important in ensuring its success. SQUID (Sequential Question and Insight Diagram) is a great model that allows a team to effectively switch between giving questions and answers and develop the skills they need to stay on track throughout the process. 

Begin with two different colored sticky notes – one for questions and one for answers – and with your central topic (the head of the squid) on the board. Ask the group to first come up with a series of questions connected to their best guess of how to approach the topic. Ask the group to come up with answers to those questions, fix them to the board and connect them with a line. After some discussion, go back to question mode by responding to the generated answers or other points on the board.

It’s rewarding to see a diagram grow throughout the exercise, and a completed SQUID can provide a visual resource for future effort and as an example for other teams.

SQUID   #gamestorming   #project planning   #issue analysis   #problem solving   When exploring an information space, it’s important for a group to know where they are at any given time. By using SQUID, a group charts out the territory as they go and can navigate accordingly. SQUID stands for Sequential Question and Insight Diagram.

16. Speed Boat

To continue with our nautical theme, Speed Boat is a short and sweet activity that can help a team quickly identify what employees, clients or service users might have a problem with and analyze what might be standing in the way of achieving a solution.

Methods that allow for a group to make observations, have insights and obtain those eureka moments quickly are invaluable when trying to solve complex problems.

In Speed Boat, the approach is to first consider what anchors and challenges might be holding an organization (or boat) back. Bonus points if you are able to identify any sharks in the water and develop ideas that can also deal with competitors!   

Speed Boat   #gamestorming   #problem solving   #action   Speedboat is a short and sweet way to identify what your employees or clients don’t like about your product/service or what’s standing in the way of a desired goal.

17. The Journalistic Six

Some of the most effective ways of solving problems is by encouraging teams to be more inclusive and diverse in their thinking.

Based on the six key questions journalism students are taught to answer in articles and news stories, The Journalistic Six helps create teams to see the whole picture. By using who, what, when, where, why, and how to facilitate the conversation and encourage creative thinking, your team can make sure that the problem identification and problem analysis stages of the are covered exhaustively and thoughtfully. Reporter’s notebook and dictaphone optional.

The Journalistic Six – Who What When Where Why How   #idea generation   #issue analysis   #problem solving   #online   #creative thinking   #remote-friendly   A questioning method for generating, explaining, investigating ideas.

18. LEGO Challenge

Now for an activity that is a little out of the (toy) box. LEGO Serious Play is a facilitation methodology that can be used to improve creative thinking and problem-solving skills. 

The LEGO Challenge includes giving each member of the team an assignment that is hidden from the rest of the group while they create a structure without speaking.

What the LEGO challenge brings to the table is a fun working example of working with stakeholders who might not be on the same page to solve problems. Also, it’s LEGO! Who doesn’t love LEGO! 

LEGO Challenge   #hyperisland   #team   A team-building activity in which groups must work together to build a structure out of LEGO, but each individual has a secret “assignment” which makes the collaborative process more challenging. It emphasizes group communication, leadership dynamics, conflict, cooperation, patience and problem solving strategy.

19. What, So What, Now What?

If not carefully managed, the problem identification and problem analysis stages of the problem-solving process can actually create more problems and misunderstandings.

The What, So What, Now What? problem-solving activity is designed to help collect insights and move forward while also eliminating the possibility of disagreement when it comes to identifying, clarifying, and analyzing organizational or work problems. 

Facilitation is all about bringing groups together so that might work on a shared goal and the best problem-solving strategies ensure that teams are aligned in purpose, if not initially in opinion or insight.

Throughout the three steps of this game, you give everyone on a team to reflect on a problem by asking what happened, why it is important, and what actions should then be taken. 

This can be a great activity for bringing our individual perceptions about a problem or challenge and contextualizing it in a larger group setting. This is one of the most important problem-solving skills you can bring to your organization.

W³ – What, So What, Now What?   #issue analysis   #innovation   #liberating structures   You can help groups reflect on a shared experience in a way that builds understanding and spurs coordinated action while avoiding unproductive conflict. It is possible for every voice to be heard while simultaneously sifting for insights and shaping new direction. Progressing in stages makes this practical—from collecting facts about What Happened to making sense of these facts with So What and finally to what actions logically follow with Now What . The shared progression eliminates most of the misunderstandings that otherwise fuel disagreements about what to do. Voila!

20. Journalists  

Problem analysis can be one of the most important and decisive stages of all problem-solving tools. Sometimes, a team can become bogged down in the details and are unable to move forward.

Journalists is an activity that can avoid a group from getting stuck in the problem identification or problem analysis stages of the process.

In Journalists, the group is invited to draft the front page of a fictional newspaper and figure out what stories deserve to be on the cover and what headlines those stories will have. By reframing how your problems and challenges are approached, you can help a team move productively through the process and be better prepared for the steps to follow.

Journalists   #vision   #big picture   #issue analysis   #remote-friendly   This is an exercise to use when the group gets stuck in details and struggles to see the big picture. Also good for defining a vision.

Problem-solving techniques for developing solutions 

The success of any problem-solving process can be measured by the solutions it produces. After you’ve defined the issue, explored existing ideas, and ideated, it’s time to narrow down to the correct solution.

Use these problem-solving techniques when you want to help your team find consensus, compare possible solutions, and move towards taking action on a particular problem.

• Improved Solutions
• Four-Step Sketch
• 15% Solutions
• How-Now-Wow matrix
• Impact Effort Matrix

21. Mindspin  

Brainstorming is part of the bread and butter of the problem-solving process and all problem-solving strategies benefit from getting ideas out and challenging a team to generate solutions quickly. 

With Mindspin, participants are encouraged not only to generate ideas but to do so under time constraints and by slamming down cards and passing them on. By doing multiple rounds, your team can begin with a free generation of possible solutions before moving on to developing those solutions and encouraging further ideation. 

This is one of our favorite problem-solving activities and can be great for keeping the energy up throughout the workshop. Remember the importance of helping people become engaged in the process – energizing problem-solving techniques like Mindspin can help ensure your team stays engaged and happy, even when the problems they’re coming together to solve are complex. 

MindSpin   #teampedia   #idea generation   #problem solving   #action   A fast and loud method to enhance brainstorming within a team. Since this activity has more than round ideas that are repetitive can be ruled out leaving more creative and innovative answers to the challenge.

22. Improved Solutions

After a team has successfully identified a problem and come up with a few solutions, it can be tempting to call the work of the problem-solving process complete. That said, the first solution is not necessarily the best, and by including a further review and reflection activity into your problem-solving model, you can ensure your group reaches the best possible result. 

One of a number of problem-solving games from Thiagi Group, Improved Solutions helps you go the extra mile and develop suggested solutions with close consideration and peer review. By supporting the discussion of several problems at once and by shifting team roles throughout, this problem-solving technique is a dynamic way of finding the best solution. 

Improved Solutions   #creativity   #thiagi   #problem solving   #action   #team   You can improve any solution by objectively reviewing its strengths and weaknesses and making suitable adjustments. In this creativity framegame, you improve the solutions to several problems. To maintain objective detachment, you deal with a different problem during each of six rounds and assume different roles (problem owner, consultant, basher, booster, enhancer, and evaluator) during each round. At the conclusion of the activity, each player ends up with two solutions to her problem.

23. Four Step Sketch

Creative thinking and visual ideation does not need to be confined to the opening stages of your problem-solving strategies. Exercises that include sketching and prototyping on paper can be effective at the solution finding and development stage of the process, and can be great for keeping a team engaged. 

By going from simple notes to a crazy 8s round that involves rapidly sketching 8 variations on their ideas before then producing a final solution sketch, the group is able to iterate quickly and visually. Problem-solving techniques like Four-Step Sketch are great if you have a group of different thinkers and want to change things up from a more textual or discussion-based approach.

Four-Step Sketch   #design sprint   #innovation   #idea generation   #remote-friendly   The four-step sketch is an exercise that helps people to create well-formed concepts through a structured process that includes: Review key information Start design work on paper,  Consider multiple variations , Create a detailed solution . This exercise is preceded by a set of other activities allowing the group to clarify the challenge they want to solve. See how the Four Step Sketch exercise fits into a Design Sprint

24. 15% Solutions

Some problems are simpler than others and with the right problem-solving activities, you can empower people to take immediate actions that can help create organizational change. 

Part of the liberating structures toolkit, 15% solutions is a problem-solving technique that focuses on finding and implementing solutions quickly. A process of iterating and making small changes quickly can help generate momentum and an appetite for solving complex problems.

Problem-solving strategies can live and die on whether people are onboard. Getting some quick wins is a great way of getting people behind the process.   

It can be extremely empowering for a team to realize that problem-solving techniques can be deployed quickly and easily and delineate between things they can positively impact and those things they cannot change. 

15% Solutions   #action   #liberating structures   #remote-friendly   You can reveal the actions, however small, that everyone can do immediately. At a minimum, these will create momentum, and that may make a BIG difference.  15% Solutions show that there is no reason to wait around, feel powerless, or fearful. They help people pick it up a level. They get individuals and the group to focus on what is within their discretion instead of what they cannot change.  With a very simple question, you can flip the conversation to what can be done and find solutions to big problems that are often distributed widely in places not known in advance. Shifting a few grains of sand may trigger a landslide and change the whole landscape.

25. How-Now-Wow Matrix

The problem-solving process is often creative, as complex problems usually require a change of thinking and creative response in order to find the best solutions. While it’s common for the first stages to encourage creative thinking, groups can often gravitate to familiar solutions when it comes to the end of the process. 

When selecting solutions, you don’t want to lose your creative energy! The How-Now-Wow Matrix from Gamestorming is a great problem-solving activity that enables a group to stay creative and think out of the box when it comes to selecting the right solution for a given problem.

Problem-solving techniques that encourage creative thinking and the ideation and selection of new solutions can be the most effective in organisational change. Give the How-Now-Wow Matrix a go, and not just for how pleasant it is to say out loud. 

How-Now-Wow Matrix   #gamestorming   #idea generation   #remote-friendly   When people want to develop new ideas, they most often think out of the box in the brainstorming or divergent phase. However, when it comes to convergence, people often end up picking ideas that are most familiar to them. This is called a ‘creative paradox’ or a ‘creadox’. The How-Now-Wow matrix is an idea selection tool that breaks the creadox by forcing people to weigh each idea on 2 parameters.

26. Impact and Effort Matrix

All problem-solving techniques hope to not only find solutions to a given problem or challenge but to find the best solution. When it comes to finding a solution, groups are invited to put on their decision-making hats and really think about how a proposed idea would work in practice. 

The Impact and Effort Matrix is one of the problem-solving techniques that fall into this camp, empowering participants to first generate ideas and then categorize them into a 2×2 matrix based on impact and effort.

Activities that invite critical thinking while remaining simple are invaluable. Use the Impact and Effort Matrix to move from ideation and towards evaluating potential solutions before then committing to them. 

Impact and Effort Matrix   #gamestorming   #decision making   #action   #remote-friendly   In this decision-making exercise, possible actions are mapped based on two factors: effort required to implement and potential impact. Categorizing ideas along these lines is a useful technique in decision making, as it obliges contributors to balance and evaluate suggested actions before committing to them.

27. Dotmocracy

If you’ve followed each of the problem-solving steps with your group successfully, you should move towards the end of your process with heaps of possible solutions developed with a specific problem in mind. But how do you help a group go from ideation to putting a solution into action? 

Dotmocracy – or Dot Voting -is a tried and tested method of helping a team in the problem-solving process make decisions and put actions in place with a degree of oversight and consensus. 

One of the problem-solving techniques that should be in every facilitator’s toolbox, Dot Voting is fast and effective and can help identify the most popular and best solutions and help bring a group to a decision effectively. 

Dotmocracy   #action   #decision making   #group prioritization   #hyperisland   #remote-friendly   Dotmocracy is a simple method for group prioritization or decision-making. It is not an activity on its own, but a method to use in processes where prioritization or decision-making is the aim. The method supports a group to quickly see which options are most popular or relevant. The options or ideas are written on post-its and stuck up on a wall for the whole group to see. Each person votes for the options they think are the strongest, and that information is used to inform a decision.

All facilitators know that warm-ups and icebreakers are useful for any workshop or group process. Problem-solving workshops are no different.

Use these problem-solving techniques to warm up a group and prepare them for the rest of the process. Activating your group by tapping into some of the top problem-solving skills can be one of the best ways to see great outcomes from your session.

• Check-in/Check-out
• Doodling Together
• Show and Tell
• Constellations
• Draw a Tree

28. Check-in / Check-out

Solid processes are planned from beginning to end, and the best facilitators know that setting the tone and establishing a safe, open environment can be integral to a successful problem-solving process.

Check-in / Check-out is a great way to begin and/or bookend a problem-solving workshop. Checking in to a session emphasizes that everyone will be seen, heard, and expected to contribute. 

If you are running a series of meetings, setting a consistent pattern of checking in and checking out can really help your team get into a groove. We recommend this opening-closing activity for small to medium-sized groups though it can work with large groups if they’re disciplined!

Check-in / Check-out   #team   #opening   #closing   #hyperisland   #remote-friendly   Either checking-in or checking-out is a simple way for a team to open or close a process, symbolically and in a collaborative way. Checking-in/out invites each member in a group to be present, seen and heard, and to express a reflection or a feeling. Checking-in emphasizes presence, focus and group commitment; checking-out emphasizes reflection and symbolic closure.

29. Doodling Together  

Thinking creatively and not being afraid to make suggestions are important problem-solving skills for any group or team, and warming up by encouraging these behaviors is a great way to start. 

Doodling Together is one of our favorite creative ice breaker games – it’s quick, effective, and fun and can make all following problem-solving steps easier by encouraging a group to collaborate visually. By passing cards and adding additional items as they go, the workshop group gets into a groove of co-creation and idea development that is crucial to finding solutions to problems. 

Doodling Together   #collaboration   #creativity   #teamwork   #fun   #team   #visual methods   #energiser   #icebreaker   #remote-friendly   Create wild, weird and often funny postcards together & establish a group’s creative confidence.

30. Show and Tell

You might remember some version of Show and Tell from being a kid in school and it’s a great problem-solving activity to kick off a session.

Asking participants to prepare a little something before a workshop by bringing an object for show and tell can help them warm up before the session has even begun! Games that include a physical object can also help encourage early engagement before moving onto more big-picture thinking.

By asking your participants to tell stories about why they chose to bring a particular item to the group, you can help teams see things from new perspectives and see both differences and similarities in the way they approach a topic. Great groundwork for approaching a problem-solving process as a team! 

Show and Tell   #gamestorming   #action   #opening   #meeting facilitation   Show and Tell taps into the power of metaphors to reveal players’ underlying assumptions and associations around a topic The aim of the game is to get a deeper understanding of stakeholders’ perspectives on anything—a new project, an organizational restructuring, a shift in the company’s vision or team dynamic.

31. Constellations

Who doesn’t love stars? Constellations is a great warm-up activity for any workshop as it gets people up off their feet, energized, and ready to engage in new ways with established topics. It’s also great for showing existing beliefs, biases, and patterns that can come into play as part of your session.

Using warm-up games that help build trust and connection while also allowing for non-verbal responses can be great for easing people into the problem-solving process and encouraging engagement from everyone in the group. Constellations is great in large spaces that allow for movement and is definitely a practical exercise to allow the group to see patterns that are otherwise invisible. 

Constellations   #trust   #connection   #opening   #coaching   #patterns   #system   Individuals express their response to a statement or idea by standing closer or further from a central object. Used with teams to reveal system, hidden patterns, perspectives.

32. Draw a Tree

Problem-solving games that help raise group awareness through a central, unifying metaphor can be effective ways to warm-up a group in any problem-solving model.

Draw a Tree is a simple warm-up activity you can use in any group and which can provide a quick jolt of energy. Start by asking your participants to draw a tree in just 45 seconds – they can choose whether it will be abstract or realistic. 

Once the timer is up, ask the group how many people included the roots of the tree and use this as a means to discuss how we can ignore important parts of any system simply because they are not visible.

All problem-solving strategies are made more effective by thinking of problems critically and by exposing things that may not normally come to light. Warm-up games like Draw a Tree are great in that they quickly demonstrate some key problem-solving skills in an accessible and effective way.

Draw a Tree   #thiagi   #opening   #perspectives   #remote-friendly   With this game you can raise awarness about being more mindful, and aware of the environment we live in.

Each step of the problem-solving workshop benefits from an intelligent deployment of activities, games, and techniques. Bringing your session to an effective close helps ensure that solutions are followed through on and that you also celebrate what has been achieved.

Here are some problem-solving activities you can use to effectively close a workshop or meeting and ensure the great work you’ve done can continue afterward.

• One Breath Feedback
• Who What When Matrix
• Response Cards

How do I conclude a problem-solving process?

All good things must come to an end. With the bulk of the work done, it can be tempting to conclude your workshop swiftly and without a moment to debrief and align. This can be problematic in that it doesn’t allow your team to fully process the results or reflect on the process.

At the end of an effective session, your team will have gone through a process that, while productive, can be exhausting. It’s important to give your group a moment to take a breath, ensure that they are clear on future actions, and provide short feedback before leaving the space. 

The primary purpose of any problem-solving method is to generate solutions and then implement them. Be sure to take the opportunity to ensure everyone is aligned and ready to effectively implement the solutions you produced in the workshop.

Remember that every process can be improved and by giving a short moment to collect feedback in the session, you can further refine your problem-solving methods and see further success in the future too.

33. One Breath Feedback

Maintaining attention and focus during the closing stages of a problem-solving workshop can be tricky and so being concise when giving feedback can be important. It’s easy to incur “death by feedback” should some team members go on for too long sharing their perspectives in a quick feedback round. 

One Breath Feedback is a great closing activity for workshops. You give everyone an opportunity to provide feedback on what they’ve done but only in the space of a single breath. This keeps feedback short and to the point and means that everyone is encouraged to provide the most important piece of feedback to them. 

One breath feedback   #closing   #feedback   #action   This is a feedback round in just one breath that excels in maintaining attention: each participants is able to speak during just one breath … for most people that’s around 20 to 25 seconds … unless of course you’ve been a deep sea diver in which case you’ll be able to do it for longer.

34. Who What When Matrix 

Matrices feature as part of many effective problem-solving strategies and with good reason. They are easily recognizable, simple to use, and generate results.

The Who What When Matrix is a great tool to use when closing your problem-solving session by attributing a who, what and when to the actions and solutions you have decided upon. The resulting matrix is a simple, easy-to-follow way of ensuring your team can move forward. 

Great solutions can’t be enacted without action and ownership. Your problem-solving process should include a stage for allocating tasks to individuals or teams and creating a realistic timeframe for those solutions to be implemented or checked out. Use this method to keep the solution implementation process clear and simple for all involved. 

Who/What/When Matrix   #gamestorming   #action   #project planning   With Who/What/When matrix, you can connect people with clear actions they have defined and have committed to.

35. Response cards

Group discussion can comprise the bulk of most problem-solving activities and by the end of the process, you might find that your team is talked out! 

Providing a means for your team to give feedback with short written notes can ensure everyone is head and can contribute without the need to stand up and talk. Depending on the needs of the group, giving an alternative can help ensure everyone can contribute to your problem-solving model in the way that makes the most sense for them.

Response Cards is a great way to close a workshop if you are looking for a gentle warm-down and want to get some swift discussion around some of the feedback that is raised. 

Response Cards   #debriefing   #closing   #structured sharing   #questions and answers   #thiagi   #action   It can be hard to involve everyone during a closing of a session. Some might stay in the background or get unheard because of louder participants. However, with the use of Response Cards, everyone will be involved in providing feedback or clarify questions at the end of a session.

Save time and effort discovering the right solutions

A structured problem solving process is a surefire way of solving tough problems, discovering creative solutions and driving organizational change. But how can you design for successful outcomes?

With SessionLab, it’s easy to design engaging workshops that deliver results. Drag, drop and reorder blocks  to build your agenda. When you make changes or update your agenda, your session  timing   adjusts automatically , saving you time on manual adjustments.

Collaborating with stakeholders or clients? Share your agenda with a single click and collaborate in real-time. No more sending documents back and forth over email.

Explore  how to use SessionLab  to design effective problem solving workshops or  watch this five minute video  to see the planner in action!

Over to you

The problem-solving process can often be as complicated and multifaceted as the problems they are set-up to solve. With the right problem-solving techniques and a mix of creative exercises designed to guide discussion and generate purposeful ideas, we hope we’ve given you the tools to find the best solutions as simply and easily as possible.

Is there a problem-solving technique that you are missing here? Do you have a favorite activity or method you use when facilitating? Let us know in the comments below, we’d love to hear from you! 

thank you very much for these excellent techniques

Certainly wonderful article, very detailed. Shared!

Leave a Comment Cancel reply

Your email address will not be published. Required fields are marked *

Going from a mere idea to a workshop that delivers results for your clients can feel like a daunting task. In this piece, we will shine a light on all the work behind the scenes and help you learn how to plan a workshop from start to finish. On a good day, facilitation can feel like effortless magic, but that is mostly the result of backstage work, foresight, and a lot of careful planning. Read on to learn a step-by-step approach to breaking the process of planning a workshop into small, manageable chunks.  The flow starts with the first meeting with a client to define the purposes of a workshop.…

How does learning work? A clever 9-year-old once told me: “I know I am learning something new when I am surprised.” The science of adult learning tells us that, in order to learn new skills (which, unsurprisingly, is harder for adults to do than kids) grown-ups need to first get into a specific headspace.  In a business, this approach is often employed in a training session where employees learn new skills or work on professional development. But how do you ensure your training is effective? In this guide, we'll explore how to create an effective training session plan and run engaging training sessions. As team leader, project manager, or consultant,…

Effective online tools are a necessity for smooth and engaging virtual workshops and meetings. But how do you choose the right ones? Do you sometimes feel that the good old pen and paper or MS Office toolkit and email leaves you struggling to stay on top of managing and delivering your workshop? Fortunately, there are plenty of online tools to make your life easier when you need to facilitate a meeting and lead workshops. In this post, we’ll share our favorite online tools you can use to make your job as a facilitator easier. In fact, there are plenty of free online workshop tools and meeting facilitation software you can…

Design your next workshop with SessionLab

Join the 150,000 facilitators using SessionLab

Sign up for free

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List

Analysing Complex Problem-Solving Strategies from a Cognitive Perspective: The Role of Thinking Skills

1 MTA-SZTE Digital Learning Technologies Research Group, Center for Learning and Instruction, University of Szeged, 6722 Szeged, Hungary

Gyöngyvér Molnár

2 MTA-SZTE Digital Learning Technologies Research Group, Institute of Education, University of Szeged, 6722 Szeged, Hungary; uh.degezs-u.yspde@ranlomyg

Associated Data

The data used to support the findings cannot be shared at this time as it also forms part of an ongoing study.

Complex problem solving (CPS) is considered to be one of the most important skills for successful learning. In an effort to explore the nature of CPS, this study aims to investigate the role of inductive reasoning (IR) and combinatorial reasoning (CR) in the problem-solving process of students using statistically distinguishable exploration strategies in the CPS environment. The sample was drawn from a group of university students (N = 1343). The tests were delivered via the eDia online assessment platform. Latent class analyses were employed to seek students whose problem-solving strategies showed similar patterns. Four qualitatively different class profiles were identified: (1) 84.3% of the students were proficient strategy users, (2) 6.2% were rapid learners, (3) 3.1% were non-persistent explorers, and (4) 6.5% were non-performing explorers. Better exploration strategy users showed greater development in thinking skills, and the roles of IR and CR in the CPS process were varied for each type of strategy user. To sum up, the analysis identified students’ problem-solving behaviours in respect of exploration strategy in the CPS environment and detected a number of remarkable differences in terms of the use of thinking skills between students with different exploration strategies.

1. Introduction

Problem solving is part and parcel of our daily activities, for instance, in determining what to wear in the morning, how to use our new electronic devices, how to reach a restaurant by public transport, how to arrange our schedule to achieve the greatest work efficiency and how to communicate with people in a foreign country. In most cases, it is essential to solve the problems that recur in our study, work and daily lives. These situations require problem solving. Generally, problem solving is the thinking that occurs if we want “to overcome barriers between a given state and a desired goal state by means of behavioural and/or cognitive, multistep activities” ( Frensch and Funke 1995, p. 18 ). It has also been considered as one of the most important skills for successful learning in the 21st century. This study focuses on one specific kind of problem solving, complex problem solving (CPS). (Numerous other terms are also used ( Funke et al. 2018 ), such as interactive problem solving ( Greiff et al. 2013 ; Wu and Molnár 2018 ), and creative problem solving ( OECD 2010 ), etc.).

CPS is a transversal skill ( Greiff et al. 2014 ), operating several mental activities and thinking skills (see Molnár et al. 2013 ). In order to explore the nature of CPS, some studies have focused on detecting its component skills ( Wu and Molnár 2018 ), whereas others have analysed students’ behaviour during the problem-solving process ( Greiff et al. 2018 ; Wu and Molnár 2021 ). This study aims to link these two fields by investigating the role of thinking skills in learning by examining students’ use of statistically distinguishable exploration strategies in the CPS environment.

1.1. Complex Problem Solving: Definition, Assessment and Relations to Intelligence

According to a widely accepted definition proposed by Buchner ( 1995 ), CPS is “the successful interaction with task environments that are dynamic (i.e., change as a function of users’ intervention and/or as a function of time) and in which some, if not all, of the environment’s regularities can only be revealed by successful exploration and integration of the information gained in that process” ( Buchner 1995, p. 14 ). A CPS process is split into two phases, knowledge acquisition and knowledge application. In the knowledge acquisition (KAC) phase of CPS, the problem solver understands the problem itself and stores the acquired information ( Funke 2001 ; Novick and Bassok 2005 ). In the knowledge application (KAP) phase, the problem solver applies the acquired knowledge to bring about the transition from a given state to a goal state ( Novick and Bassok 2005 ).

Problem solving, especially CPS, has frequently been compared or linked to intelligence in previous studies (e.g., Beckmann and Guthke 1995 ; Stadler et al. 2015 ; Wenke et al. 2005 ). Lotz et al. ( 2017 ) observed that “intelligence and [CPS] are two strongly overlapping constructs” (p. 98). There are many similarities and commonalities that can be detected between CPS and intelligence. For instance, CPS and intelligence share some of the same key features, such as the integration of information ( Stadler et al. 2015 ). Furthermore, Wenke et al. ( 2005 ) stated that “the ability to solve problems has featured prominently in virtually every definition of human intelligence” (p. 9); meanwhile, from the opposite perspective, intelligence has also been considered as one of the most important predictors of the ability to solve problems ( Wenke et al. 2005 ). Moreover, the relation between CPS and intelligence has also been discussed from an empirical perspective. A meta-analysis conducted by Stadler et al. ( 2015 ) selected 47 empirical studies (total sample size N = 13,740) which focused on the correlation between CPS and intelligence. The results of their analysis confirmed that a correlation between CPS and intelligence exists with a moderate effect size of M(g) = 0.43.

Due to the strong link between CPS and intelligence, assessments of these two domains have been connected and have overlapped to a certain extent. For instance, Beckmann and Guthke ( 1995 ) observed that some of the intelligence tests “capture something akin to an individual’s general ability to solve problems (e.g., Sternberg 1982 )” (p. 184). Nowadays, some widely used CPS assessment methods are related to intelligence but still constitute a distinct construct ( Schweizer et al. 2013 ), such as the MicroDYN approach ( Greiff and Funke 2009 ; Greiff et al. 2012 ; Schweizer et al. 2013 ). This approach uses the minimal complex system to simulate simplistic, artificial but still complex problems following certain construction rules ( Greiff and Funke 2009 ; Greiff et al. 2012 ).

The MicroDYN approach has been widely employed to measure problem solving in a well-defined problem context (i.e., “problems have a clear set of means for reaching a precisely described goal state”, Dörner and Funke 2017, p. 1 ). To complete a task based on the MicroDYN approach, the problem solver engages in dynamic interaction with the task to acquire relevant knowledge. It is not possible to create this kind of test environment with the traditional paper-and-pencil-based method. Therefore, it is currently only possible to conduct a MicroDYN-based CPS assessment within the computer-based assessment framework. In the context of computer-based assessment, the problem-solvers’ operations were recorded and logged by the assessment platform. Thus, except for regular achievement-focused result data, logfile data are also available for analysis. This provides the option of exploring and monitoring problem solvers’ behaviour and thinking processes, specifically, their exploration strategies, during the problem-solving process (see, e.g., Chen et al. 2019 ; Greiff et al. 2015a ; Molnár and Csapó 2018 ; Molnár et al. 2022 ; Wu and Molnár 2021 ).

Problem solving, in the context of an ill-defined problem (i.e., “problems have no clear problem definition, their goal state is not defined clearly, and the means of moving towards the (diffusely described) goal state are not clear”, Dörner and Funke 2017, p. 1), involved a different cognitive process than that in the context of a well-defined problem ( Funke 2010 ; Schraw et al. 1995 ), and it cannot be measured with the MicroDYN approach. The nature of ill-defined problem solving has been explored and discussed in numerous studies (e.g., Dörner and Funke 2017 ; Hołda et al. 2020 ; Schraw et al. 1995 ; Welter et al. 2017 ). This will not be discussed here as this study focuses on well-defined problem solving.

1.2. Inductive and Combinatorial Reasoning as Component Skills of Complex Problem Solving

Frensch and Funke ( 1995 ) constructed a theoretical framework that summarizes the basic components of CPS and the interrelations among the components. The framework contains three separate components: problem solver, task and environment. The impact of the problem solver is mainly relevant to three main categories, which are memory contents, dynamic information processing and non-cognitive variables. Some thinking skills have been reported to play an important role in dynamic information processing. We can thus describe them as component skills of CPS. Inductive reasoning (IR) and combinatorial reasoning (CR) are the two thinking skills that have been most frequently discussed as component skills of CPS.

IR is the reasoning skill that has been covered most commonly in the literature. Currently, there is no universally accepted definition. Molnár et al. ( 2013 ) described it as the cognitive process of acquiring general regularities by generalizing single and specific observations and experiences, whereas Klauer ( 1990 ) defined it as the discovery of regularities that relies upon the detection of similarities and/or dissimilarities as concerns attributes of or relations to or between objects. Sandberg and McCullough ( 2010 ) provided a general conclusion of the definitions of IR: it is the process of moving from the specific to the general.

Csapó ( 1997 ) pointed out that IR is a basic component of thinking and that it forms a central aspect of intellectual functioning. Some studies have also discussed the role of IR in a problem-solving environment. For instance, Mayer ( 1998 ) stated that IR will be applied in information processing during the process of solving general problems. Gilhooly ( 1982 ) also pointed out that IR plays a key role in some activities in the problem-solving process, such as hypothesis generation and hypothesis testing. Moreover, the influence of IR on both KAC and KAP has been analysed and demonstrated in previous studies ( Molnár et al. 2013 ).

Empirical studies have also provided evidence that IR and CPS are related. Based on the results of a large-scale assessment (N = 2769), Molnár et al. ( 2013 ) showed that IR significantly correlated with 9–17-year-old students’ domain-general problem-solving achievement (r = 0.44–0.52). Greiff et al. ( 2015b ) conducted a large-scale assessment project (N = 2021) in Finland to explore the links between fluid reasoning skills and domain-general CPS. The study measured fluid reasoning as a two-dimensional model which consisted of deductive reasoning and scientific reasoning and included inductive thinking processes ( Greiff et al. 2015b ). The results drawing on structural equation modelling indicated that fluid reasoning which was partly based on IR had significant and strong predictive effects on both KAC (β = 0.51) and KAP (β = 0.55), the two phases of problem solving. Such studies have suggested that IR is one of the component skills of CPS.

According to Adey and Csapó ’s ( 2012 ) definition, CR is the process of creating complex constructions out of a set of given elements that satisfy the conditions explicitly given in or inferred from the situation. In this process, some cognitive operations, such as combinations, arrangements, permutations, notations and formulae, will be employed ( English 2005 ). CR is one of the basic components of formal thinking ( Batanero et al. 1997 ). The relationship between CR and CPS has frequently been discussed. English ( 2005 ) demonstrated that CR has an essential meaning in several types of problem situations, such as problems requiring the systematic testing of alternative solutions. Moreover, Newell ( 1993 ) pointed out that CR is applied in some key activities of problem-solving information processing, such as strategy generation and application. Its functions include, but are not limited to, helping problem solvers to discover relationships between certain elements and concepts, promoting their fluency of thinking when they are considering different strategies ( Csapó 1999 ) and identifying all possible alternatives ( OECD 2014 ). Moreover, Wu and Molnár ’s ( 2018 ) empirical study drew on a sample (N = 187) of 11–13-year-old primary school students in China. Their study built a structural equation model between CPS, IR and CR, and the result indicated that CR showed a strong and statistically significant predictive power for CPS (β = 0.55). Thus, the results of the empirical study also support the argument that CR is one of the component skills of CPS.

1.3. Behaviours and Strategies in a Complex Problem-Solving Environment

Wüstenberg et al. ( 2012 ) stated that the creation and implementation of strategic exploration are core actions of the problem-solving task. Exploring and generating effective information are key to successfully solving a problem. Wittmann and Hattrup ( 2004 ) illustrated that “riskier strategies [create] a learning environment with greater opportunities to discover and master the rules and boundaries [of a problem]” (p. 406). Thus, when gathering information about a complex problem, there may be differences between exploration strategies in terms of efficacy. The MicroDYN scenarios, a simplification and simulation of the real-world problem-solving context, will also be influenced by the adoption and implementation of exploration strategies.

The effectiveness of the isolated variation strategy (or “Vary-One-Thing-At-A-Time” strategy—VOTAT; Vollmeyer et al. 1996 ) in a CPS environment has been hotly debated ( Chen et al. 2019 ; Greiff et al. 2018 ; Molnár and Csapó 2018 ; Molnár et al. 2022 ; Wu and Molnár 2021 ; Wüstenberg et al. 2014 ). To use the VOTAT strategy, a problem solver “systematically varies only one input variable, whereas the others remain unchanged. This way, the effect of the variable that has just been changed can be observed directly by monitoring the changes in the output variables” ( Molnár and Csapó 2018, p. 2 ). Understanding and using VOTAT effectively is the foundation for developing more complex strategies for coordinating multiple variables and the basis for some phases of scientific thinking (i.e., inquiry, analysis, inference and argument; Kuhn 2010 ; Kuhn et al. 1995 ).

Some previous studies have indicated that students who are able to apply VOTAT are more likely to achieve higher performance in a CPS assessment ( Greiff et al. 2018 ), especially if the problem is a well-defined minimal complex system (such as MicroDYN) ( Fischer et al. 2012 ; Molnár and Csapó 2018 ; Wu and Molnár 2021 ). For instance, Molnár and Csapó ( 2018 ) conducted an empirical study to explore how students’ exploration strategies influence their performance in an interactive problem-solving environment. They measured a group (N = 4371) of 3rd- to 12th-grade (aged 9–18) Hungarian students’ problem-solving achievement and modelled students’ exploration strategies. This result confirmed that students’ exploration strategies influence their problem-solving performance. For example, conscious VOTAT strategy users proved to be the best problem-solvers. Furthermore, other empirical studies (e.g., Molnár et al. 2022 ; Wu and Molnár 2021 ) achieved similar results, thus confirming the importance of VOTAT in a MicroDYN-based CPS environment.

Lotz et al. ( 2017 ) illustrated that effective use of VOTAT is associated with higher levels of intelligence. Their study also pointed out that intelligence has the potential to facilitate successful exploration behaviour. Reasoning skills are an important component of general intelligence. Based on Lotz et al. ’s ( 2017 ) statements, the roles IR and CR play in the CPS process might vary due to students’ different strategy usage patterns. However, there is still a lack of empirical studies in this regard.

2. Research Aims and Questions

Numerous studies have explored the nature of CPS, some of them discussing and analysing it from behavioural or cognitive perspectives. However, there have barely been any that have merged these two perspectives. From the cognitive perspective, this study explores the role of thinking skills (including IR and CR) in the cognition process of CPS. From the behavioural perspective, the study focuses on students’ behaviour (i.e., their exploration strategy) in the CPS assessment process. More specifically, the research aims to fill this gap and examine students’ use of statistically distinguishable exploration strategies in CPS environments and to detect the connection between the level of students’ thinking skills and their behaviour strategies in the CPS environment. The following research questions were thus formed.

• (RQ1) What exploration strategy profiles characterise the various problem-solvers at the university level?
• (RQ2) Can developmental differences in CPS, IR and CR be detected among students with different exploration strategy profiles?
• (RQ3) What are the similarities and differences in the roles IR and CR play in the CPS process as well as in the two phases of CPS (i.e., KAC and KAP) among students with different exploration strategy profiles?

3.1. Participants and Procedure

The sample was drawn from one of the largest universities in Hungary. Participation was voluntary, but students were able to earn one course credit for taking part in the assessment. The participants were students who had just started their studies there (N = 1671). 43.4% of the first-year students took part in the assessment. 50.9% of the participants were female, and 49.1% were male. We filtered the sample and excluded those who had more than 80% missing data on any of the tests. After the data were cleaned, data from 1343 students were available for analysis. The test was designed and delivered via the eDia online assessment system ( Csapó and Molnár 2019 ). The assessment was held in the university ICT room and divided into two sessions. The first session involved the CPS test, whereas the second session entailed the IR and CR tests. Each session lasted 45 min. The language of the tests was Hungarian, the mother tongue of the students.

3.2. Instruments

3.2.1. complex problem solving (cps).

The CPS assessment instrument adopted the MicroDYN approach. It contains a total of twelve scenarios, and each scenario consisted of two items (one item in the KAC phase and one item in the KAP phase in each problem scenario). Twelve KAC items and twelve KAP items were therefore delivered on the CPS test for a total of twenty-four items. Each scenario has a fictional cover story. For instance, students found a sick cat in front of their house, and they were expected to feed the cat with two different kinds of cat food to help it recover.

Each item contains up to three input and three output variables. The relations between the input and output variables were formulated with linear structural equations ( Funke 2001 ). Figure 1 shows a MicroDYN sample structure containing three input variables (A, B and C), three output variables (X, Y and Z) and a number of possible relations between the variables. The complexity of the item was defined by the number of input and output variables, and the number of relations between the variables. The test began with the item with the lowest complexity. The complexity of each item gradually increased as the test progressed.

A typical MicroDYN structure with three input variables and three output variables ( Greiff and Funke 2009 ).

The interface of each item displays the value of each variable in both numerical and figural forms (See Figure 2 ). Each of the input variables has a controller, which makes it possible to vary and set the value between +2 (+ +) and −2 (− −). To operate the system, students need to click the “+” or “−” button or use the slider directly to select the value they want to be added to or subtracted from the current value of the input variable. After clicking the “Apply” button in the interface, the input variables will add or subtract the selected value, and the output variables will show the corresponding changes. The history of the values for the input and output variables within the same problem scenario is displayed on screen. If students want to withdraw all the changes and set all the variables to their original status, they can click the “Reset” button.

Screenshot of the MicroDYN item Cat—first phase (knowledge acquisition). (The items were administered in Hungarian.)

In the first phase of the problem-solving process, the KAC phase, students are asked to interact with the system by changing the value of the input variables and observing and analysing the corresponding changes in the output variables. They are then expected to determine the relationship between the input and output variables and draw it in the form of (an) arrow(s) on the concept map at the bottom of the interface. To avoid item dependence in the second phase of the problem-solving process, the students are provided with a concept map during the KAP phase (see Figure 3 ), which shows the correct connections between the input and output variables. The students are expected to interact with the system by manipulating the input variables to make the output variables reach the given target values in four steps or less. That is, they cannot click on the “Apply” button more than four times. The first phase had a 180 s time limit, whereas the second had a 90 s time limit.

Screenshot of the MicroDYN item Cat—second phase (knowledge application). (The items were administered in Hungarian).

3.2.2. Inductive Reasoning (IR)

The IR instrument (see Figure 4 ) was originally designed and developed in Hungary ( Csapó 1997 ). In the last 25 years, the instrument has been further developed and scaled for a wide age range ( Molnár and Csapó 2011 ). In addition, figural items have been added, and the assessment method has evolved from paper-and-pencil to computer-based ( Pásztor 2016 ). Currently, the instrument is widely employed in a number of countries (see, e.g., Mousa and Molnár 2020 ; Pásztor et al. 2018 ; Wu et al. 2022 ; Wu and Molnár 2018 ). In the present study, four types of items were included after test adaptation: figural series, figural analogies, number analogies and number series. Students were expected to ascertain the correct relationship between the given figures and numbers and select a suitable figure or number as their answer. Students used the drag-and-drop operation to provide their answers. In total, 49 inductive reasoning items were delivered to the participating students.

Sample items for the IR test. (The items were administered in Hungarian.).

3.2.3. Combinatorial Reasoning (CR)

The CR instrument (see Figure 5 ) was originally designed by Csapó ( 1988 ). The instrument was first developed in paper-and-pencil format and then modified for computer use ( Pásztor and Csapó 2014 ). Each item contained figural or verbal elements and a clear requirement for combing through the elements. Students were asked to list every single combination based on a given rule they could find. For the figural items, students provided their answers using the drag-and-drop operation; for the verbal items, they were asked to type their answers in a text box provided on screen. The test consisted of eight combinatorial reasoning items in total.

Sample item for the CR test. (The items were administered in Hungarian).

3.3. Scoring

Students’ performance was automatically scored via the eDia platform. Items on the CPS and IR tests were scored dichotomously. In the first phase (KAC) of the CPS test, if a student drew all the correct relations on the concept map provided on screen within the given timeframe, his/her performance was assigned a score of 1 or otherwise a score of 0. In the second phase (KAP) of the CPS test, if the student successfully reached the given target values of the output variables by manipulating the level of the input variables within no more than four steps and the given timeframe, then his/her performance earned a score of 1 or otherwise a score of 0. On the IR test items, if a student selected the correct figure or number as his/her answer, then he or she received a score of 1; otherwise, the score was 0.

Students’ performance on the CR test items was scored according to a special J index, which was developed by Csapó ( 1988 ). The J index ranges from 0 to 1, where 1 means that the student provided all the correct combinations without any redundant combinations on the task. The formula for computing the J index is the following:

x stands for the number of correct combinations in the student’s answer,

T stands for the number of all possible correct combinations, and

y stands for the number of redundant combinations in the student’s answer.

Furthermore, according to Csapó ’s ( 1988 ) design, if y is higher than T, then the J index will be counted as 0.

3.4. Coding and Labelling the Logfile Data

Beyond concrete answer data, students’ interaction and manipulation behaviour were also logged in the assessment system. This made it possible to analyse students’ exploration behaviour in the first phase of the CPS process (KAC phase). Toward this aim, we adopted a labelling system developed by Molnár and Csapó ( 2018 ) to transfer the raw logfile data to structured data files for analysis. Based on the system, each trial (i.e., the sum of manipulations within the same problem scenario which was applied and tested by clicking the “Apply” button) was modelled as a single data entity. The sum of these trials within the same problem was defined as a strategy. In our study, we only consider the trials which were able to provide useful and new information for the problem-solvers, whereas the redundant or operations trials were excluded.

In this study, we analysed students’ trials to determine the extent to which they used the VOTAT strategy: fully, partially or not at all. This strategy is the most successful exploration strategy for such problems; it is the easiest to interpret and provides direct information about the given variable without any mediation effects ( Fischer et al. 2012 ; Greiff et al. 2018 ; Molnár and Csapó 2018 ; Wüstenberg et al. 2014 ; Wu and Molnár 2021 ). Based on the definition of VOTAT noted in Section 1.3 , we checked students’ trials to ascertain if they systematically varied one input variable while keeping the others unchanged, or applied a different, less successful strategy. We considered the following three types of trials:

• “Only one single input variable was manipulated, whose relationship to the output variables was unknown (we considered a relationship unknown if its effect cannot be known from previous settings), while the other variables were set at a neutral value like zero […]
• One single input variable was changed, whose relationship to the output variables was unknown. The others were not at zero, but at a setting used earlier. […]
• One single input variable was changed, whose relationship to the output variables was unknown, and the others were not at zero; however, the effect of the other input variable(s) was known from earlier settings. Even so, this combination was not attempted earlier” ( Molnár and Csapó 2018, p. 8 )

We used the numbers 0, 1 and 2 to distinguish the level of students’ use of the most effective exploration strategy (i.e., VOTAT). If a student applied one or more of the above trials for every input variable within the same scenario, we considered that they had used the full VOTAT strategy and labelled this behaviour 2. If a student had only employed VOTAT on some but not all of the input variables, we concluded that they had used a partial VOTAT strategy for that problem scenario and labelled it 1. If a student had used none of the trials noted above in their problem exploration, then we determined that they had not used VOTAT at all and thus gave them a label of 0.

3.5. Data Analysis Plan

We used LCA (latent class analysis) to explore students’ exploration strategy profiles. LCA is a latent variable modelling approach that can be used to identify unmeasured (latent) classes of samples with similarly observed variables. LCA has been widely used in analysing logfile data for CPS assessment and in exploring students’ behaviour patterns (see, e.g., Gnaldi et al. 2020 ; Greiff et al. 2018 ; Molnár et al. 2022 ; Molnár and Csapó 2018 ; Mustafić et al. 2019 ; Wu and Molnár 2021 ). The scores for the use of VOTAT in the KAC phase (0, 1, 2; see Section 3.4 ) were used for the LCA analysis. We used Mplus ( Muthén and Muthén 2010 ) to run the LCA analysis. Several indices were used to measure the model fit: AIC (Akaike information criterion), BIC (Bayesian information criterion) and aBIC (adjusted Bayesian information criterion). With these three indicators, lower values indicate a better model fit. Entropy (ranging from 0 to 1, with values close to 1 indicating high certainty in the classification). The Lo–Mendell–Rubin adjusted likelihood ratio was used to compare the model containing n latent classes with the model containing n − 1 latent classes, and the p value was the indicator for whether a significant difference could be detected ( Lo et al. 2001 ). The results of the Lo–Mendell–Rubin adjusted likelihood ratio analysis were used to decide the correct number of latent classes in LCA models.

ANOVA was used to analyse the performance differences for CPS, IR and CR across the students from the different class profiles. The analysis was run using SPSS. A path analysis (PA) was employed in the structural equation modelling (SEM) framework to investigate the roles of CR and IR in CPS and the similarities and differences across the students from the different exploration strategy profiles. The PA models were carried out with Mplus. The Tucker–Lewis index (TLI), the comparative fit index (CFI) and the root-mean-square error of approximation (RMSEA) were used as indicators for the model fit. A TLI and CFI larger than 0.90 paired with a RMSEA less than 0.08 are commonly considered as an acceptable model fit ( van de Schoot et al. 2012 ).

4.1. Descriptive Results

All three tests showed good reliability (Cronbach’s α: CPS: 0.89; IR: 0.87; CR: 0.79). Furthermore, the two sub-dimensions of the CPS test, KAC and KAP, also showed satisfactory reliability (Cronbach’s α: KAC: 0.86; KAP: 0.78). The tests thus proved to be reliable. The means and standard deviations of students’ performance (in percentage) on each test are provided in Table 1 .

The means and standard deviations of students’ performance on each test.

4.2. Four Qualitatively Different Exploration Strategy Profiles Can Be Distinguished in CPS

Based on the labelled logfile data for CPS, we applied latent class analyses to identify the behaviour patterns of the students in the exploration phase of the problem-solving process. The model fits for the LCA analysis are listed in Table 2 . Compared with the 2 or 3 latent class models, the 4 latent class model has a lower AIC, BIC and aBIC, and the likelihood ratio statistical test (the Lo–Mendell–Rubin adjusted likelihood ratio test) confirmed it has a significantly better model fit. The 5 and 6 latent class models did not show a better model fit than the 4 latent class model. Therefore, based on the results, four qualitatively different exploration strategy profiles can be distinguished, which covered 96% of the students.

Fit indices for latent class analyses.

The patterns for the four qualitatively different exploration strategy profiles are shown in Figure 6 . In total, 84.3% of the students were proficient exploration strategy users, who were able to use VOTAT in each problem scenario independent of its difficulty level (represented by the red line in Figure 5 ). In total, 6.2% of the students were rapid learners. They were not able to apply VOTAT at the beginning of the test on the easiest problems but managed to learn quickly, and, after a rapid learning curve by the end of the test, they reached the level of proficient exploration strategy users, even though the problems became much more complex (represented by the blue line). In total, 3.1% of the students proved to be non-persistent explorers, and they employed VOTAT on the easiest problems but did not transfer this knowledge to the more complex problems. Finally, they were no longer able to apply VOTAT when the complexity of the problems increased (represented by the green line). In total, 6.5% of the students were non-performing explorers; they barely used any VOTAT strategy during the whole test (represented by the pink line) independent of problem complexity.

Four qualitatively different exploration strategy profiles.

4.3. Better Exploration Strategy Users Showed Better Performance in Reasoning Skills

Students with different exploration strategy profiles showed different kinds of performance in each reasoning skill under investigation. Results (see Table 3 ) showed that more proficient strategy users tended to have higher achievement in all the domains assessed as well as in the two sub-dimensions in CPS (i.e., KAC and KAP; ANOVA: CPS: F(3, 1339) = 187.28, p < 0.001; KAC: F(3, 1339) = 237.15, p < 0.001; KAP: F(3, 1339) = 74.91, p < 0.001; IR: F(3, 1339) = 48.10, p < 0.001; CR: F(3, 1339) = 28.72, p < 0.001); specifically, students identified as “proficient exploration strategy users” achieved the highest level on the reasoning skills tests independent of the domains. On average, they were followed by rapid learners, non-persistent explorers and, finally, non-performing explorers. Tukey’s post hoc tests revealed more details on the performance differences of students with different exploration profiles in each of the domains being measured. Proficient strategy users proved to be significantly more skilled in each of the reasoning domains. They were followed by rapid learners, who outperformed non-persistent explorers and non-performing explorers in CPS. In the domains of IR and CR, there were no achievement differences between rapid learners and non-persistent explorers, who significantly outperformed non-performing strategy explorers.

Students’ performance on each test—grouped according to the different exploration strategy profiles.

4.4. The Roles of IR and CR in CPS and Its Processes Were Different for Each Type of Exploration Strategy User

Path analysis was used to explore the predictive power of IR and CR for CPS and its processes, knowledge acquisition and knowledge application, for each group of students with different exploration strategy profiles. That is, four path analysis models were built to indicate the predictive power of IR and CR for CPS (see Figure 7 ), and another four path analyses models were developed to monitor the predictive power of IR and CR for the two empirically distinguishable phases of CPS (i.e., KAC and KAP) (see Figure 8 ). All eight models had good model fits, the fit indices TLI and CFI were above 0.90, and RMSEA was less than 0.08.

Path analysis models (with CPS, IR and CR) for each type of strategy user; * significant at 0.05 ( p   <  0.05); ** significant at 0.01 ( p   <  0.01); N.S.: no significant effect can be found.

Path analysis models (with KAC, KAP, IR and CR) for each type of strategy user; * significant at 0.05 ( p  <  0.05); ** significant at 0.01 ( p  <  0.01); N.S.: no significant effect can be found.

Students’ level of IR significantly predicted their level of CPS in all four path analysis models independent of their exploration strategy profile ( Figure 7 ; proficient strategy users: β = 0.432, p < 0.01; rapid learners: β = 0.350, p < 0.01; non-persistent explorers: β = 0.309, p < 0.05; and non-performing explorers: β = 0.386, p < 0.01). This was not the case for CR, which only proved to have predictive power for CPS among proficient strategy users (β = 0.104, p < 0.01). IR and CR were significantly correlated in all four models.

After examining the roles of IR and CR in the CPS process, we went further to explore the roles of these two reasoning skills in the distinguishable phases of CPS. The path analysis models ( Figure 8 ) showed that the predictive power of IR and CR for KAC and KAP was varied in each group. Levels of IR and CR among non-persistent explorers and non-performing explorers failed to predict their achievement in the KAC phase of the CPS process. Moreover, rapid learners’ level of IR significantly predicted their achievement in the KAC phase (β = 0.327, p < 0.01), but their level of CR did not have the same predictive power. Furthermore, the proficient strategy users’ levels of both reasoning skills had significant predictive power for KAC (IR: β = 0.363, p < 0.01; CR: β = 0.132, p < 0.01). In addition, in the KAP phase of the CPS problems, IR played a significant role for all types of strategy users, although with different power (proficient strategy users: β = 0.408, p < 0.01; rapid learners: β = 0.339, p < 0.01; non-persistent explorers: β = 0.361, p < 0.01; and non-performing explorers: β = 0.447, p < 0.01); by contrast, CR did not have significant predictive power for the KAP phase in any of the models.

5. Discussion

The study aims to investigate the role of IR and CR in CPS and its phases among students using statistically distinguishable exploration strategies in different CPS environments. We examined 1343 Hungarian university students and assessed their CPS, IR and CR skills. Both achievement data and logfile data were used in the analysis. The traditional achievement indicators formed the foundation for analysing the students’ CPS, CR and IR performance, whereas process data extracted from logfile data were used to explore students’ exploration behaviour in various CPS environments.

Four qualitatively different exploration strategy profiles were distinguished: proficient strategy users, rapid learners, non-persistent explorers and non-performing explorers (RQ1). The four profiles were consistent with the result of another study conducted at university level (see Molnár et al. 2022 ), and the frequencies of these four profiles in these two studies were very similar. The two studies therefore corroborate and validate each other’s results. The majority of the participants were identified as proficient strategy users. More than 80% of the university students were able to employ effective exploration strategies in various CPS environments. Of the remaining students, some performed poorly in exploration strategy use in the early part of the test (rapid learners), some in the last part (non-persistent explorers) and some throughout the test (non-performing explorers). However, students with these three exploration strategy profiles only constituted small portions of the total sample (with proportions ranging from 3.1% to 6.5%). The university students therefore exhibited generally good performance in terms of exploration strategy use in a CPS environment, especially compared with previous results among younger students (e.g., primary school students, see Greiff et al. 2018 ; Wu and Molnár 2021 ; primary to secondary students, see Molnár and Csapó 2018 ).

The results have indicated that better exploration strategy users achieved higher CPS performance and had better development levels of IR and CR (RQ2). First, the results have confirmed the importance of VOTAT in a CPS environment. This finding is consistent with previous studies (e.g., Greiff et al. 2015a ; Molnár and Csapó 2018 ; Mustafić et al. 2019 ; Wu and Molnár 2021 ). Second, the results have confirmed that effective use of VOTAT is strongly tied to the level of IR and CR development. Reasoning forms an important component of human intelligence, and the level of development in reasoning was an indicator of the level of intelligence ( Klauer et al. 2002 ; Sternberg and Kaufman 2011 ). Therefore, this finding has supplemented empirical evidence for the argument that effective use of VOTAT is associated with levels of intelligence to a certain extent.

The roles of IR and CR proved to be varied for each type of exploration strategy user (RQ3). For instance, the level of CPS among the best exploration strategy users (i.e., the proficient strategy users) was predicted by both the levels of IR and CR, but this was not the case for students with other profiles. In addition, the results have indicated that IR played important roles in both the KAC and KAP phases for the students with relatively good exploration strategy profiles (i.e., proficient strategy users and rapid learners) but only in the KAP phase for the rest of the students (non-persistent explorers and non-performing explorers); moreover, the predictive power of CR can only be detected in the KAC phase of the proficient strategy users. To sum up, the results suggest a general trend of IR and CR playing more important roles in the CPS process among better exploration strategy users.

Combining the answers to RQ2 and RQ3, we can gain further insights into students’ exploration strategy use in a CPS environment. Our results have confirmed that the use of VOTAT is associated with the level of IR and CR development and that the importance of IR and CR increases with proficiency in exploration strategy use. Based on these findings, we can make a reasonable argument that IR and CR are essential skills for using VOTAT and that underdeveloped IR and CR will prevent students from using effective strategies in a CPS environment. Therefore, if we want to encourage students to become better exploration strategy users, it is important to first enhance their IR and CR skills. Previous studies have suggested that establishing explicit training in using effective strategies in a CPS environment is important for students’ CPS development ( Molnár et al. 2022 ). Our findings have identified the importance of IR and CR in exploration strategy use, which has important implications for designing training programmes.

The results have also provided a basis for further studies. Future studies have been suggested to further link the behavioural and cognitive perspectives in CPS research. For instance, IR and CR were considered as component skills of CPS (see Section 1.2 ). The results of the study have indicated the possibility of not only discussing the roles of IR and CR in the cognitive process of CPS, but also exploration behaviour in a CPS environment. The results have thus provided a new perspective for exploring the component skills of CPS.

6. Limitations

There are some limitations in the study. All the tests were low stake; therefore, students might not be sufficiently motivated to do their best. This feature might have produced the missing values detected in the sample. In addition, some students’ exploration behaviour shown in this study might theoretically be below their true level. However, considering that data cleaning was adopted in this study (see Section 3.1 ), we believe this phenomenon will not have a remarkable influence on the results. Moreover, the CPS test in this study was based on the MicroDYN approach, which is a well-established and widely used artificial model with a limited number of variables and relations. However, it does not have the power to cover all kinds of complex and dynamic problems in real life. For instance, the MicroDYN approach cannot measure ill-defined problem solving. Thus, this study can only demonstrate the influence of IR and CR on problem solving in well-defined MicroDYN-simulated problems. Furthermore, VOTAT is helpful with minimally complex problems under well-defined laboratory conditions, but it may not be that helpful with real-world, ill-defined complex problems ( Dörner and Funke 2017 ; Funke 2021 ). Therefore, the generalizability of the findings is limited.

7. Conclusions

In general, the results have shed new light on students’ problem-solving behaviours in respect of exploration strategy in a CPS environment and explored differences in terms of the use of thinking skills between students with different exploration strategies. Most studies discuss students’ problem-solving strategies from a behavioural perspective. By contrast, this paper discusses them from both behavioural and cognitive perspectives, thus expanding our understanding in this area. As for educational implications, the study contributes to designing and revising training methods for CPS by identifying the importance of IR and CR in exploration behaviour in a CPS environment. To sum up, the study has investigated the nature of CPS from a fresh angle and provided a sound basis for future studies.

Funding Statement

This study has been conducted with support provided by the National Research, Development and Innovation Fund of Hungary, financed under the OTKA K135727 funding scheme and supported by the Research Programme for Public Education Development, Hungarian Academy of Sciences (KOZOKT2021-16).

Author Contributions

Conceptualization, H.W. and G.M.; methodology, H.W. and G.M.; formal analysis, H.W.; writing—original draft preparation, H.W.; writing—review and editing, G.M.; project administration, G.M.; funding acquisition, G.M. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Ethical approval was not required for this study in accordance with the national and institutional guidelines. The assessments which provided data for this study were integrated parts of the educational processes of the participating university. The participation was voluntary.

Informed Consent Statement

All of the students in the assessment turned 18, that is, it was not required or possible to request and obtain written informed parental consent from the participants.

Data Availability Statement

Conflicts of interest.

Authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Adey Philip, Csapó Benő. Developing and Assessing Scientific Reasoning. In: Csapó Benő, Szabó Gábor., editors. Framework for Diagnostic Assessment of Science. Nemzeti Tankönyvkiadó; Budapest: 2012. pp. 17–53. [ Google Scholar ]
• Batanero Carmen, Navarro-Pelayo Virginia, Godino Juan D. Effect of the implicit combinatorial model on combinatorial reasoning in secondary school pupils. Educational Studies in Mathematics. 1997; 32 :181–99. doi: 10.1023/A:1002954428327. [ CrossRef ] [ Google Scholar ]
• Beckmann Jens F., Guthke Jürgen. Complex problem solving, intelligence, and learning ability. In: Frensch Peter A., Funke Joachim., editors. Complex Problem Solving: The European Perspective. Erlbaum; Hillsdale: 1995. pp. 177–200. [ Google Scholar ]
• Buchner Axel. Basic topics and approaches to the study of complex problem solving. In: Frensch Peter A., Funke Joachim., editors. Complex Problem Solving: The European Perspective. Erlbaum; Hillsdale: 1995. pp. 27–63. [ Google Scholar ]
• Chen Yunxiao, Li Xiaoou, Liu Jincheng, Ying Zhiliang. Statistical analysis of complex problem-solving process data: An event history analysis approach. Frontiers in Psychology. 2019; 10 :486. doi: 10.3389/fpsyg.2019.00486. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Csapó Benő. A kombinatív képesség struktúrája és fejlődése. Akadémiai Kiadó; Budapest: 1988. [ Google Scholar ]
• Csapó Benő. The development of inductive reasoning: Cross-sectional assessments in an educational context. International Journal of Behavioral Development. 1997; 20 :609–26. doi: 10.1080/016502597385081. [ CrossRef ] [ Google Scholar ]
• Csapó Benő. Teaching and Learning Thinking Skills. Swets & Zeitlinger; Lisse: 1999. Improving thinking through the content of teaching; pp. 37–62. [ Google Scholar ]
• Csapó Benő, Molnár Gyöngyvér. Online diagnostic assessment in support of personalized teaching and learning: The eDia System. Frontiers in Psychology. 2019; 10 :1522. doi: 10.3389/fpsyg.2019.01522. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Dörner Dietrich, Funke Joachim. Complex problem solving: What it is and what it is not. Frontiers in Psychology. 2017; 8 :1153. doi: 10.3389/fpsyg.2017.01153. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• English Lyn D. Combinatorics and the development of children’s combinatorial reasoning. In: Jones Graham A., editor. Exploring Probability in School: Challenges for Teaching and Learning. Springer; New York: 2005. pp. 121–41. [ Google Scholar ]
• Fischer Andreas, Greiff Samuel, Funke Joachim. The process of solving complex problems. Journal of Problem Solving. 2012; 4 :19–42. doi: 10.7771/1932-6246.1118. [ CrossRef ] [ Google Scholar ]
• Frensch Peter A., Funke Joachim. Complex Problem Solving: The European Perspective. Psychology Press; New York: 1995. [ Google Scholar ]
• Funke Joachim. Dynamic systems as tools for analysing human judgement. Thinking and Reasoning. 2001; 7 :69–89. doi: 10.1080/13546780042000046. [ CrossRef ] [ Google Scholar ]
• Funke Joachim. Complex problem solving: A case for complex cognition? Cognitive Processing. 2010; 11 :133–42. doi: 10.1007/s10339-009-0345-0. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Funke Joachim. It Requires More Than Intelligence to Solve Consequential World Problems. Journal of Intelligence. 2021; 9 :38. doi: 10.3390/jintelligence9030038. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Funke Joachim, Fischer Andreas, Holt Daniel V. Competencies for complexity: Problem solving in the twenty-first century. In: Care Esther, Griffin Patrick, Wilson Mark., editors. Assessment and Teaching of 21st Century Skills. Springer; Dordrecht: 2018. pp. 41–53. [ Google Scholar ]
• Gilhooly Kenneth J. Thinking: Directed, Undirected and Creative. Academic Press; London: 1982. [ Google Scholar ]
• Gnaldi Michela, Bacci Silvia, Kunze Thiemo, Greiff Samuel. Students’ complex problem solving profiles. Psychometrika. 2020; 85 :469–501. doi: 10.1007/s11336-020-09709-2. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Greiff Samuel, Funke Joachim. Measuring complex problem solving-the MicroDYN approach. In: Scheuermann Friedrich, Björnsson Julius., editors. The Transition to Computer-Based Assessment. Office for Official Publications of the European Communities; Luxembourg: 2009. pp. 157–63. [ Google Scholar ]
• Greiff Samuel, Holt Daniel V., Funke Joachim. Perspectives on problem solving in educational assessment: Analytical, interactive, and collaborative problem solving. Journal of Problem Solving. 2013; 5 :71–91. doi: 10.7771/1932-6246.1153. [ CrossRef ] [ Google Scholar ]
• Greiff Samuel, Molnár Gyöngyvér, Martina Romain, Zimmermann Johannes, Csapó Benő. Students’ exploration strategies in computer-simulated complex problem environments: A latent class approach. Computers & Education. 2018; 126 :248–63. [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Avvisati Francesco. Computer-generated log-file analyses as a window into students’ minds? A showcase study based on the PISA 2012 assessment of problem solving. Computers & Education. 2015a; 91 :92–105. [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Funke Joachim. Dynamic problem solving: A new measurement perspective. Applied Psychological Measurement. 2012; 36 :189–213. doi: 10.1177/0146621612439620. [ CrossRef ] [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Csapó Benő, Demetriou Andreas, Hautamäki Jarkko, Graesser Arthur C., Martin Romain. Domain-general problem solving skills and education in the 21st century. Educational Research Review. 2014; 13 :74–83. doi: 10.1016/j.edurev.2014.10.002. [ CrossRef ] [ Google Scholar ]
• Greiff Samuel, Wüstenberg Sascha, Goetz Thomas, Vainikainen Mari-Pauliina, Hautamäki Jarkko, Bornstein Marc H. A longitudinal study of higher-order thinking skills: Working memory and fluid reasoning in childhood enhance complex problem solving in adolescence. Frontiers in Psychology. 2015b; 6 :1060. doi: 10.3389/fpsyg.2015.01060. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hołda Małgorzata, Głodek Anna, Dankiewicz-Berger Malwina, Skrzypińska Dagna, Szmigielska Barbara. Ill-defined problem solving does not benefit from daytime napping. Frontiers in Psychology. 2020; 11 :559. doi: 10.3389/fpsyg.2020.00559. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Klauer Karl Josef. Paradigmatic teaching of inductive thinking. Learning and Instruction. 1990; 2 :23–45. [ Google Scholar ]
• Klauer Karl Josef, Willmes Klaus, Phye Gary D. Inducing inductive reasoning: Does it transfer to fluid intelligence? Contemporary Educational Psychology. 2002; 27 :1–25. doi: 10.1006/ceps.2001.1079. [ CrossRef ] [ Google Scholar ]
• Kuhn Deanna. What is scientific thinking and how does it develop? In: Goswami Usha., editor. The Wiley-Blackwell Handbook of Childhood Cognitive Development. Wiley-Blackwell; Oxford: 2010. pp. 371–93. [ Google Scholar ]
• Kuhn Deanna, Garcia-Mila Merce, Zohar Anat, Andersen Christopher, Sheldon H. White, Klahr David, Carver Sharon M. Strategies of knowledge acquisition. Monographs of the Society for Research in Child Development. 1995; 60 :1–157. doi: 10.2307/1166059. [ CrossRef ] [ Google Scholar ]
• Lo Yungtai, Mendell Nancy R., Rubin Donald B. Testing the number of components in a normal mixture. Biometrika. 2001; 88 :767–78. doi: 10.1093/biomet/88.3.767. [ CrossRef ] [ Google Scholar ]
• Lotz Christin, Scherer Ronny, Greiff Samuel, Sparfeldt Jörn R. Intelligence in action—Effective strategic behaviors while solving complex problems. Intelligence. 2017; 64 :98–112. doi: 10.1016/j.intell.2017.08.002. [ CrossRef ] [ Google Scholar ]
• Mayer Richard E. Cognitive, metacognitive, and motivational aspects of problem solving. Instructional Science. 1998; 26 :49–63. doi: 10.1023/A:1003088013286. [ CrossRef ] [ Google Scholar ]
• Molnár Gyöngyvér, Csapó Benő. Az 1–11 évfolyamot átfogó induktív gondolkodás kompetenciaskála készítése a valószínűségi tesztelmélet alkalmazásával. Magyar Pedagógia. 2011; 111 :127–40. [ Google Scholar ]
• Molnár Gyöngyvér, Csapó Benő. The efficacy and development of students’ problem-solving strategies during compulsory schooling: Logfile analyses. Frontiers in Psychology. 2018; 9 :302. doi: 10.3389/fpsyg.2018.00302. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Molnár Gyöngyvér, Alrababah Saleh Ahmad, Greiff Samuel. How we explore, interpret, and solve complex problems: A cross-national study of problem-solving processes. Heliyon. 2022; 8 :e08775. doi: 10.1016/j.heliyon.2022.e08775. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Molnár Gyöngyvér, Greiff Samuel, Csapó Benő. Inductive reasoning, domain specific and complex problem solving: Relations and development. Thinking Skills and Creativity. 2013; 9 :35–45. doi: 10.1016/j.tsc.2013.03.002. [ CrossRef ] [ Google Scholar ]
• Mousa Mojahed, Molnár Gyöngyvér. Computer-based training in math improves inductive reasoning of 9- to 11-year-old children. Thinking Skills and Creativity. 2020; 37 :100687. doi: 10.1016/j.tsc.2020.100687. [ CrossRef ] [ Google Scholar ]
• Mustafić Maida, Yu Jing, Stadler Matthias, Vainikainen Mari-Pauliina, Bornstein Marc H., Putnick Diane L., Greiff Samuel. Complex problem solving: Profiles and developmental paths revealed via latent transition analysis. Developmental Psychology. 2019; 55 :2090–101. doi: 10.1037/dev0000764. [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Muthén Linda K., Muthén Bengt O. Mplus User’s Guide. Muthén & Muthén; Los Angeles: 2010. [ Google Scholar ]
• Newell Allen. Reasoning, Problem Solving, and Decision Processes: The Problem Space as a Fundamental Category. MIT Press; Boston: 1993. [ Google Scholar ]
• Novick Laura R., Bassok Miriam. Problem solving. In: Holyoak Keith James, Morrison Robert G., editors. The Cambridge Handbook of Thinking and Reasoning. Cambridge University Press; New York: 2005. pp. 321–49. [ Google Scholar ]
• OECD . PISA 2012 Field Trial Problem Solving Framework. OECD Publishing; Paris: 2010. [ Google Scholar ]
• OECD . Results: Creative Problem Solving—Students’ Skills in Tackling Real-Life Problems (Volume V) OECD Publishing; Paris: 2014. [ Google Scholar ]
• Pásztor Attila. Ph.D. thesis. Doctoral School of Education, University of Szeged; Szeged, Hungary: 2016. Technology-Based Assessment and Development of Inductive Reasoning. [ Google Scholar ]
• Pásztor Attila, Csapó Benő. Improving Combinatorial Reasoning through Inquiry-Based Science Learning; Paper presented at the Science and Mathematics Education Conference; Dublin, Ireland. June 24–25; 2014. [ Google Scholar ]
• Pásztor Attila, Kupiainen Sirkku, Hotulainen Risto, Molnár Gyöngyvér, Csapó Benő. Comparing Finnish and Hungarian Fourth Grade Students’ Inductive Reasoning Skills; Paper presented at the EARLI SIG 1 Conference; Helsinki, Finland. August 29–31; 2018. [ Google Scholar ]
• Sandberg Elisabeth Hollister, McCullough Mary Beth. The development of reasoning skills. In: Sandberg Elisabeth Hollister, Spritz Becky L., editors. A Clinician’s Guide to Normal Cognitive Development in Childhood. Routledge; New York: 2010. pp. 179–89. [ Google Scholar ]
• Schraw Gregory, Dunkle Michael E., Bendixen Lisa D. Cognitive processes in well-defined and ill-defined problem solving. Applied Cognitive Psychology. 1995; 9 :523–38. doi: 10.1002/acp.2350090605. [ CrossRef ] [ Google Scholar ]
• Schweizer Fabian, Wüstenberg Sascha, Greiff Samuel. Validity of the MicroDYN approach: Complex problem solving predicts school grades beyond working memory capacity. Learning and Individual Differences. 2013; 24 :42–52. doi: 10.1016/j.lindif.2012.12.011. [ CrossRef ] [ Google Scholar ]
• Stadler Matthias, Becker Nicolas, Gödker Markus, Leutner Detlev, Greiff Samuel. Complex problem solving and intelligence: A meta-analysis. Intelligence. 2015; 53 :92–101. doi: 10.1016/j.intell.2015.09.005. [ CrossRef ] [ Google Scholar ]
• Sternberg Robert J. Handbook of Human Intelligence. Cambridge University Press; New York: 1982. [ Google Scholar ]
• Sternberg Robert J., Kaufman Scott Barry. The Cambridge Handbook of Intelligence. Cambridge University Press; New York: 2011. [ Google Scholar ]
• van de Schoot Rens, Lugtig Peter, Hox Joop. A checklist for testing measurement invariance. European Journal of Developmental Psychology. 2012; 9 :486–92. doi: 10.1080/17405629.2012.686740. [ CrossRef ] [ Google Scholar ]
• Vollmeyer Regina, Burns Bruce D., Holyoak Keith J. The impact of goal specificity on strategy use and the acquisition of problem structure. Cognitive Science. 1996; 20 :75–100. doi: 10.1207/s15516709cog2001_3. [ CrossRef ] [ Google Scholar ]
• Welter Marisete Maria, Jaarsveld Saskia, Lachmann Thomas. Problem space matters: The development of creativity and intelligence in primary school children. Creativity Research Journal. 2017; 29 :125–32. doi: 10.1080/10400419.2017.1302769. [ CrossRef ] [ Google Scholar ]
• Wenke Dorit, Frensch Peter A., Funke Joachim. Complex Problem Solving and intelligence: Empirical relation and causal direction. In: Sternberg Robert J., Pretz Jean E., editors. Cognition and Intelligence: Identifying the Mechanisms of the Mind. Cambridge University Press; New York: 2005. pp. 160–87. [ Google Scholar ]
• Wittmann Werner W., Hattrup Keith. The relationship between performance in dynamic systems and intelligence. Systems Research and Behavioral Science. 2004; 21 :393–409. doi: 10.1002/sres.653. [ CrossRef ] [ Google Scholar ]
• Wu Hao, Molnár Gyöngyvér. Interactive problem solving: Assessment and relations to combinatorial and inductive reasoning. Journal of Psychological and Educational Research. 2018; 26 :90–105. [ Google Scholar ]
• Wu Hao, Molnár Gyöngyvér. Logfile analyses of successful and unsuccessful strategy use in complex problem-solving: A cross-national comparison study. European Journal of Psychology of Education. 2021; 36 :1009–32. doi: 10.1007/s10212-020-00516-y. [ CrossRef ] [ Google Scholar ]
• Wu Hao, Saleh Andi Rahmat, Molnár Gyöngyvér. Inductive and combinatorial reasoning in international educational context: Assessment, measurement invariance, and latent mean differences. Asia Pacific Education Review. 2022; 23 :297–310. doi: 10.1007/s12564-022-09750-z. [ CrossRef ] [ Google Scholar ]
• Wüstenberg Sascha, Greiff Samuel, Funke Joachim. Complex problem solving—More than reasoning? Intelligence. 2012; 40 :1–14. doi: 10.1016/j.intell.2011.11.003. [ CrossRef ] [ Google Scholar ]
• Wüstenberg Sascha, Greiff Samuel, Molnár Gyöngyvér, Funke Joachim. Cross-national gender differences in complex problem solving and their determinants. Learning and Individual Differences. 2014; 29 :18–29. doi: 10.1016/j.lindif.2013.10.006. [ CrossRef ] [ Google Scholar ]

How To Solve Complex Problems

In today’s increasingly complex world, we are constantly faced with ill-defined problems that don’t have a clear solution. From poverty and climate change to crime and addiction, complex situations surround us. Unlike simple problems with a pre-defined or “right” answer, complex problems share several basic characteristics that make them hard to solve. While these problems can be frustrating and overwhelming, they also offer an opportunity for growth and creativity. Complex problem-solving skills are the key to addressing these tough issues.

In this article, I will discuss simple versus complex problems, define complex problem solving, and describe why it is so important in complex dynamic environments. I will also explain how to develop problem-solving skills and share some tips for effectively solving complex problems.

How is simple problem-solving different from complex problem-solving?

Solving problems is about getting from a currently undesirable state to an intended goal state. In other words, about bridging the gap between “what is” and “what ought to be”. However, the challenge of reaching a solution varies based on the kind of problem that is being solved. There are generally three different kinds of problems you should consider.

Simple problems have one problem solution. The goal is to find that answer as quickly and efficiently as possible. Puzzles are classic examples of simple problem solving. The objective is to find the one correct solution out of many possibilities.

Problems are different from puzzles in that they don’t have a known problem solution. As such, many people may agree that there is an issue to be solved, but they may not agree on the intended goal state or how to get there. In this type of problem, people spend a lot of time debating the best solution and the optimal way to achieve it.

Messes are collections of interrelated problems where many stakeholders may not even agree on what the issue is. Unlike problems where there is agreement about what the problem is, in messes, there isn’t agreement amongst stakeholders. In other words, even “what is” can’t be taken for granted. Most complex social problems are messes, made up of interrelated social issues with ill-defined boundaries and goals.

Problems and messes can be complicated or complex

Puzzles are simple, but problems and messes exist on a continuum between complicated and complex. Complicated problems are technical in nature. There may be many involved variables, but the relationships are linear. As a result, complicated problems have step-by-step, systematic solutions. Repairing an engine or building a rocket may be difficult because of the many parts involved, but it is a technical problem we call complicated.

On the other hand, solving a complex problem is entirely different. Unlike complicated problems that may have many variables with linear relationships, a complex problem is characterized by connectivity patterns that are harder to understand and predict.

Characteristics of complex problems and messes

So what else makes a problem complex? Here are seven additional characteristics (from Funke and Hester and Adams ).

• Lack of information. There is often a lack of data or information about the problem itself. In some cases, variables are unknown or cannot be measured.
• Many goals. A complex problem has a mix of conflicting objectives. In some sense, every stakeholder involved with the problem may have their own goals. However, with limited resources, not all goals can be simultaneously satisfied.
• Unpredictable feedback loops. In part due to many variables connected by a range of different relationships, a change in one variable is likely to have effects on other variables in the system. However, because we do not know all of the variables it will affect, small changes can have disproportionate system-wide effects. These unexpected events that have big, unpredictable effects are sometimes called Black Swans.
• Dynamic. A complex problem changes over time and there is a significant impact based on when you act. In other words, because the problem and its parts and relationships are constantly changing, an action taken today won’t have the same effects as the same action taken tomorrow.
• Time-delayed. It takes a while for cause and effect to be realized. Thus it is very hard to know if any given intervention is working.
• Unknown unknowns. Building off the previous point about a lack of information, in a complex problem you may not even know what you don’t know. In other words, there may be very important variables that you are not even aware of.
• Affected by (error-prone) humans. Simply put, human behavior tends to be illogical and unpredictable. When humans are involved in a problem, avoiding error may be impossible.

What is complex problem-solving?

“Complex problem solving” is the term for how to address a complex problem or messes that have the characteristics listed above.

Since a complex problem is a different phenomenon than a simple or complicated problem, solving them requires a different approach. Methods designed for simple problems, like systematic organization, deductive logic, and linear thinking don’t work well on their own for a complex problem.

And yet, despite its importance, there isn’t complete agreement about what exactly it is.

How is complex problem solving defined by experts?

Let’s look at what scientists, researchers, and system thinkers have come up with in terms of a definition for solving a complex problem. 

As a series of observations and informed decisions

For many employers, the focus is on making smart decisions. These must weigh the future effects to the company of any given solution. According to Indeed.com , it is defined as “a series of observations and informed decisions used to find and implement a solution to a problem. Beyond finding and implementing a solution, complex problem solving also involves considering future changes to circumstance, resources, and capabilities that may affect the trajectory of the process and success of the solution. Complex problem solving also involves considering the impact of the solution on the surrounding environment and individuals.”

As using information to review options and develop solutions

For others, it is more of a systematic way to consider a range of options. According to O*NET ,  the definition focuses on “identifying complex problems and reviewing related information to develop and evaluate options and implement solutions.”

As a self-regulated psychological process

Others emphasize the broad range of skills and emotions needed for change. In addition, they endorse an inspired kind of pragmatism. For example, Dietrich Dorner and Joachim Funke define it as “a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.”

As a novel way of thinking and reasoning

Finally, some emphasize the multidisciplinary nature of knowledge and processes needed to tackle a complex problem. Patrick Hester and Kevin MacG. Adams have stated that “no single discipline can solve truly complex problems. Problems of real interest, those vexing ones that keep you up at night, require a discipline-agnostic approach…Simply they require us to think systemically about our problem…a novel way of thinking and reasoning about complex problems that encourages increased understanding and deliberate intervention.”

A synthesis definition

By pulling the main themes of these definitions together, we can get a sense of what complex problem-solvers must do:

Gain a better understanding of the phenomena of a complex problem or mess. Use a discipline-agnostic approach in order to develop deliberate interventions. Take into consideration future impacts on the surrounding environment.

Why is complex problem solving important?

Many efforts aimed at complex social problems like reducing homelessness and improving public health – despite good intentions giving more effort than ever before – are destined to fail because their approach is based on simple problem-solving. And some efforts might even unwittingly be contributing to the problems they’re trying to solve. 

Einstein said that “We can’t solve problems by using the same kind of thinking we used when we created them.” I think he could have easily been alluding to the need for more complex problem solvers who think differently. So what skills are required to do this?

What are complex problem-solving skills?

The skills required to solve a complex problem aren’t from one domain, nor are they an easily-packaged bundle. Rather, I like to think of them as a balancing act between a series of seemingly opposite approaches but synthesized. This brings a sort of cognitive dissonance into the process, which is itself informative.

It brings F. Scott Fitzgerald’s maxim to mind: 

“The test of a first-rate intelligence is the ability to hold two opposing ideas in mind at the same time and still retain the ability to function. One should, for example, be able to see that things are hopeless yet be determined to make them otherwise.” 

To see the problem situation clearly, for example, but also with a sense of optimism and possibility.

Here are the top three dialectics to keep in mind:

Thinking and reasoning

Reasoning is the ability to make logical deductions based on evidence and counterevidence. On the other hand, thinking is more about imagining an unknown reality based on thoughts about the whole picture and how the parts could fit together. By thinking clearly, one can have a sense of possibility that prepares the mind to deduce the right action in the unique moment at hand.

As Dorner and Funke explain: “Not every situation requires the same action,  and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.”

Analysis and reductionism combined with synthesis and holism

It’s important to be able to use scientific processes to break down a complex problem into its parts and analyze them. But at the same time, a complex problem is more than the sum of its parts. In most cases, the relationships between the parts are more important than the parts themselves. Therefore, decomposing problems with rigor isn’t enough. What’s needed, once problems are reduced and understood, is a way of understanding the relationships between various components as well as putting the pieces back together. However, synthesis and holism on their own without deductive analysis can often miss details and relationships that matter.  

What makes this balancing act more difficult is that certain professions tend to be trained in and prefer one domain over the other. Scientists prefer analysis and reductionism whereas most social scientists and practitioners default to synthesis and holism. Unfortunately, this divide of preferences results in people working in their silos at the expense of multi-disciplinary approaches that together can better “see” complexity.

Situational awareness and self-awareness 

Dual awareness is the ability to pay attention to two experiences simultaneously. In the case of complex problems, context really matters. In other words, problem-solving exists in an ecosystem of environmental factors that are not incidental. Personal and cultural preferences play a part as do current events unfolding over time. But as a problem solver, knowing the environment is only part of the equation. 

The other crucial part is the internal psychological process unique to every individual who also interacts with the problem and the environment. Problem solvers inevitably come into contact with others who may disagree with them, or be advancing seemingly counterproductive solutions, and these interactions result in emotions and motivations. Without self-awareness, we can become attached to our own subjective opinions, fall in love with “our” solutions, and generally be driven by the desire to be seen as problem solvers at the expense of actually solving the problem.

By balancing these three dialectics, practitioners can better deal with uncertainty as well as stay motivated despite setbacks. Self-regulation among these seemingly opposite approaches also reminds one to stay open-minded.

How do you develop complex problem-solving skills?

There is no one answer to this question, as the best way to develop them will vary depending on your strengths and weaknesses. However, there are a few general things that you can do to improve your ability to solve problems.

Ground yourself in theory and knowledge

First, it is important to learn about systems thinking and complexity theories. These frameworks will help you understand how complex systems work, and how different parts of a system interact with each other. This conceptual understanding will allow you to identify potential solutions to problems more quickly and effectively.

Practice switching between approaches

Second, practice switching between the dialectics mentioned above. For example, in your next meeting try to spend roughly half your time thinking and half your time reasoning. The important part is trying to get habituated to regularly switching lenses. It may seem disjointed at first, but after a while, it becomes second nature to simultaneously see how the parts interact and the big picture.

Focus on the specific problem phenomena

Third, it may sound obvious, but people often don’t spend very much time studying the problem itself and how it functions. In some sense, becoming a good problem-solver involves becoming a problem scientist. Your time should be spent regularly investigating the phenomena of “what is” rather than “what ought to be”. A holistic understanding of the problem is the required prerequisite to coming up with good solutions.

Stay curious

Finally, after we have worked on a problem for a while, we tend to think we know everything about it, including how to solve it. Even if we’re working on a problem, which may change dynamically from day to day, we start treating it more like a puzzle with a definite solution. When that happens, we can lose our motivation to continue learning about the problem. This is very risky because it closes the door to learning from others, regardless of whether we completely agree with them or not.

As Neils Bohr said, “Two different perspectives or models about a system will reveal truths regarding the system that are neither entirely independent nor entirely compatible.”

By staying curious, we can retain our ability to learn on a daily basis.

Tips for how to solve complex problems

Focus on processes over results.

It’s easy to get lost in utopian thinking. Many people spend so much time on “what ought to be” that they forget that problem solving is about the gap between “what is” and “what ought to be”. It is said that “life is a journey, not a destination.” The same is true for complex problem-solving. To do it well, a problem solver must focus on enjoying the process of gaining a holistic understanding of the problem. 

Adaptive and iterative methods and tools

A variety of adaptive and iterative methods have been developed to address complexity. They share a laser focus on gaining holistic understanding with tools that best match the phenomena of complexity. They are also non-ideological, trans-disciplinary, and flexible. In most cases, your journey through a set of steps won’t be linear. Rather, as you think and reason, analyze and synthesize, you’ll jump around to get a holistic picture.

In my online course , we generally follow a seven-step method:

• Get clear sight with a complex problem-solving frame
• Establish a secure base of operation
• Gain a deep understanding of the problem
• Create an interactive model of the problem
• Develop an impact strategy
• Create an action plan and implement
• Embed systemic solutions

Of course, each of these steps involves testing to see what works and consistently evaluating our process and progress.

Resolution is about systematically managing a problem over time

One last thing to keep in mind. Most social problems are not just solved one day, never to return. In reality,  most complex problems are managed, not solved. For all practical purposes, what this means is that “the solution” is a way of systematically dealing with the problem over time. Some find this disappointing, but it’s actually a pragmatic pointer to think about resolution – a way move problems in the right direction – rather than final solutions.

Problem solvers regularly train and practice

If you need help developing your complex problem-solving skills, I have an online class where you can learn everything you need to know. 

Sign up today and learn how to be successful at making a difference in the world!

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Complex Problem Solving

Find solutions to real-world problems

Complex Problem Solving is the skill of applying a method to a problem, often not seen before, to obtain a satisfactory solution. It requires a creative combination of knowledge and strategies to arrive at an answer. Rapid technological change, the increasingly global exchange of ideas, and the proliferation of easy-to-access information – some of which is decidedly unreliable – all contribute greater complexity to the problems that they will need to solve.

Excelling in Complex Problem Solving as a Job Skill means:

• Engaged in ‘big picture’ thinking
• Flexible & adaptable to change
• Highly detail-oriented
• Someone who sees patterns
• Someone who works efficiently

Others see you:

• Demonstrate self-reliance
• Achieve your dreams and ambitions
• Capable of higher-order thinking (not just memorizing facts, but demonstrating the ability to deeply understand, apply, analyze, and evaluate information)
• Achieve increased status & responsibility at school or work
• Create solutions that balance the facts, but with new insight

“ 4.9 Complex Problem Solving ” from Working in Play: Planning for a Career in the Recreation and Leisure Industry in Canada  by Linda Whitehead, BA, M Ed, MBA is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

The skill of applying a method to a problem, often not seen before, to obtain a satisfactory solution; it requires a creative combination of knowledge and strategies to arrive at an answer.

Fanshawe SOAR Copyright © 2023 by Kristen Cavanagh is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

• Bipolar Disorder
• Therapy Center
• When To See a Therapist
• Types of Therapy
• Best Online Therapy
• Best Couples Therapy
• Best Family Therapy
• Managing Stress
• Sleep and Dreaming
• Understanding Emotions
• Self-Improvement
• Healthy Relationships
• Student Resources
• Personality Types
• Guided Meditations
• Verywell Mind Insights
• 2023 Verywell Mind 25
• Mental Health in the Classroom
• Editorial Process
• Meet Our Review Board
• Crisis Support

Overview of the Problem-Solving Mental Process

Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Rachel Goldman, PhD FTOS, is a licensed psychologist, clinical assistant professor, speaker, wellness expert specializing in eating behaviors, stress management, and health behavior change.

• Identify the Problem
• Define the Problem
• Form a Strategy
• Organize Information
• Allocate Resources
• Monitor Progress
• Evaluate the Results

Frequently Asked Questions

Problem-solving is a mental process that involves discovering, analyzing, and solving problems. The ultimate goal of problem-solving is to overcome obstacles and find a solution that best resolves the issue.

The best strategy for solving a problem depends largely on the unique situation. In some cases, people are better off learning everything they can about the issue and then using factual knowledge to come up with a solution. In other instances, creativity and insight are the best options.

It is not necessary to follow problem-solving steps sequentially, It is common to skip steps or even go back through steps multiple times until the desired solution is reached.

In order to correctly solve a problem, it is often important to follow a series of steps. Researchers sometimes refer to this as the problem-solving cycle. While this cycle is portrayed sequentially, people rarely follow a rigid series of steps to find a solution.

The following steps include developing strategies and organizing knowledge.

1. Identifying the Problem

While it may seem like an obvious step, identifying the problem is not always as simple as it sounds. In some cases, people might mistakenly identify the wrong source of a problem, which will make attempts to solve it inefficient or even useless.

Some strategies that you might use to figure out the source of a problem include :

• Asking questions about the problem
• Breaking the problem down into smaller pieces
• Looking at the problem from different perspectives
• Conducting research to figure out what relationships exist between different variables

2. Defining the Problem

After the problem has been identified, it is important to fully define the problem so that it can be solved. You can define a problem by operationally defining each aspect of the problem and setting goals for what aspects of the problem you will address

At this point, you should focus on figuring out which aspects of the problems are facts and which are opinions. State the problem clearly and identify the scope of the solution.

3. Forming a Strategy

After the problem has been identified, it is time to start brainstorming potential solutions. This step usually involves generating as many ideas as possible without judging their quality. Once several possibilities have been generated, they can be evaluated and narrowed down.

The next step is to develop a strategy to solve the problem. The approach used will vary depending upon the situation and the individual's unique preferences. Common problem-solving strategies include heuristics and algorithms.

• Heuristics are mental shortcuts that are often based on solutions that have worked in the past. They can work well if the problem is similar to something you have encountered before and are often the best choice if you need a fast solution.
• Algorithms are step-by-step strategies that are guaranteed to produce a correct result. While this approach is great for accuracy, it can also consume time and resources.

Heuristics are often best used when time is of the essence, while algorithms are a better choice when a decision needs to be as accurate as possible.

4. Organizing Information

Before coming up with a solution, you need to first organize the available information. What do you know about the problem? What do you not know? The more information that is available the better prepared you will be to come up with an accurate solution.

When approaching a problem, it is important to make sure that you have all the data you need. Making a decision without adequate information can lead to biased or inaccurate results.

5. Allocating Resources

Of course, we don't always have unlimited money, time, and other resources to solve a problem. Before you begin to solve a problem, you need to determine how high priority it is.

If it is an important problem, it is probably worth allocating more resources to solving it. If, however, it is a fairly unimportant problem, then you do not want to spend too much of your available resources on coming up with a solution.

At this stage, it is important to consider all of the factors that might affect the problem at hand. This includes looking at the available resources, deadlines that need to be met, and any possible risks involved in each solution. After careful evaluation, a decision can be made about which solution to pursue.

6. Monitoring Progress

After selecting a problem-solving strategy, it is time to put the plan into action and see if it works. This step might involve trying out different solutions to see which one is the most effective.

It is also important to monitor the situation after implementing a solution to ensure that the problem has been solved and that no new problems have arisen as a result of the proposed solution.

Effective problem-solvers tend to monitor their progress as they work towards a solution. If they are not making good progress toward reaching their goal, they will reevaluate their approach or look for new strategies .

7. Evaluating the Results

After a solution has been reached, it is important to evaluate the results to determine if it is the best possible solution to the problem. This evaluation might be immediate, such as checking the results of a math problem to ensure the answer is correct, or it can be delayed, such as evaluating the success of a therapy program after several months of treatment.

Once a problem has been solved, it is important to take some time to reflect on the process that was used and evaluate the results. This will help you to improve your problem-solving skills and become more efficient at solving future problems.

A Word From Verywell​

It is important to remember that there are many different problem-solving processes with different steps, and this is just one example. Problem-solving in real-world situations requires a great deal of resourcefulness, flexibility, resilience, and continuous interaction with the environment.

Get Advice From The Verywell Mind Podcast

Hosted by therapist Amy Morin, LCSW, this episode of The Verywell Mind Podcast shares how you can stop dwelling in a negative mindset.

Follow Now : Apple Podcasts / Spotify / Google Podcasts

You can become a better problem solving by:

• Practicing brainstorming and coming up with multiple potential solutions to problems
• Being open-minded and considering all possible options before making a decision
• Breaking down problems into smaller, more manageable pieces
• Asking for help when needed
• Researching different problem-solving techniques and trying out new ones
• Learning from mistakes and using them as opportunities to grow

It's important to communicate openly and honestly with your partner about what's going on. Try to see things from their perspective as well as your own. Work together to find a resolution that works for both of you. Be willing to compromise and accept that there may not be a perfect solution.

Take breaks if things are getting too heated, and come back to the problem when you feel calm and collected. Don't try to fix every problem on your own—consider asking a therapist or counselor for help and insight.

If you've tried everything and there doesn't seem to be a way to fix the problem, you may have to learn to accept it. This can be difficult, but try to focus on the positive aspects of your life and remember that every situation is temporary. Don't dwell on what's going wrong—instead, think about what's going right. Find support by talking to friends or family. Seek professional help if you're having trouble coping.

Davidson JE, Sternberg RJ, editors.  The Psychology of Problem Solving .  Cambridge University Press; 2003. doi:10.1017/CBO9780511615771

Sarathy V. Real world problem-solving .  Front Hum Neurosci . 2018;12:261. Published 2018 Jun 26. doi:10.3389/fnhum.2018.00261

By Kendra Cherry, MSEd Kendra Cherry, MS, is a psychosocial rehabilitation specialist, psychology educator, and author of the "Everything Psychology Book."

Problem Solving Skills for the Digital Age

Lucid Content

Reading time: about 6 min

Let’s face it: Things don’t always go according to plan. Systems fail, wires get crossed, projects fall apart.

Problems are an inevitable part of life and work. They’re also an opportunity to think critically and find solutions. But knowing how to get to the root of unexpected situations or challenges can mean the difference between moving forward and spinning your wheels.

Here, we’ll break down the key elements of problem solving, some effective problem solving approaches, and a few effective tools to help you arrive at solutions more quickly.

So, what is problem solving?

Broadly defined, problem solving is the process of finding solutions to difficult or complex issues. But you already knew that. Understanding problem solving frameworks, however, requires a deeper dive.

Think about a recent problem you faced. Maybe it was an interpersonal issue. Or it could have been a major creative challenge you needed to solve for a client at work. How did you feel as you approached the issue? Stressed? Confused? Optimistic? Most importantly, which problem solving techniques did you use to tackle the situation head-on? How did you organize thoughts to arrive at the best possible solution?

Solve your problem-solving problem  

Here’s the good news: Good problem solving skills can be learned. By its nature, problem solving doesn’t adhere to a clear set of do’s and don’ts—it requires flexibility, communication, and adaptation. However, most problems you face, at work or in life, can be tackled using four basic steps.

First, you must define the problem . This step sounds obvious, but often, you can notice that something is amiss in a project or process without really knowing where the core problem lies. The most challenging part of the problem solving process is uncovering where the problem originated.

Second, you work to generate alternatives to address the problem directly. This should be a collaborative process to ensure you’re considering every angle of the issue.

Third, you evaluate and test potential solutions to your problem. This step helps you fully understand the complexity of the issue and arrive at the best possible solution.

Finally, fourth, you select and implement the solution that best addresses the problem.

Following this basic four-step process will help you approach every problem you encounter with the same rigorous critical and strategic thinking process, recognize commonalities in new problems, and avoid repeating past mistakes.

In addition to these basic problem solving skills, there are several best practices that you should incorporate. These problem solving approaches can help you think more critically and creatively about any problem:

You may not feel like you have the right expertise to resolve a specific problem. Don’t let that stop you from tackling it. The best problem solvers become students of the problem at hand. Even if you don’t have particular expertise on a topic, your unique experience and perspective can lend itself to creative solutions.

Challenge the status quo

Standard problem solving methodologies and problem solving frameworks are a good starting point. But don’t be afraid to challenge assumptions and push boundaries. Good problem solvers find ways to apply existing best practices into innovative problem solving approaches.

Think broadly about and visualize the issue

Sometimes it’s hard to see a problem, even if it’s right in front of you. Clear answers could be buried in rows of spreadsheet data or lost in miscommunication. Use visualization as a problem solving tool to break down problems to their core elements. Visuals can help you see bottlenecks in the context of the whole process and more clearly organize your thoughts as you define the problem.  

Hypothesize, test, and try again

It might be cliche, but there’s truth in the old adage that 99% of inspiration is perspiration. The best problem solvers ask why, test, fail, and ask why again. Whether it takes one or 1,000 iterations to solve a problem, the important part—and the part that everyone remembers—is the solution.

Consider other viewpoints

Today’s problems are more complex, more difficult to solve, and they often involve multiple disciplines. They require group expertise and knowledge. Being open to others’ expertise increases your ability to be a great problem solver. Great solutions come from integrating your ideas with those of others to find a better solution. Excellent problem solvers build networks and know how to collaborate with other people and teams. They are skilled in bringing people together and sharing knowledge and information.

4 effective problem solving tools

As you work through the problem solving steps, try these tools to better define the issue and find the appropriate solution.

Root cause analysis

Similar to pulling weeds from your garden, if you don’t get to the root of the problem, it’s bound to come back. A root cause analysis helps you figure out the root cause behind any disruption or problem, so you can take steps to correct the problem from recurring. The root cause analysis process involves defining the problem, collecting data, and identifying causal factors to pinpoint root causes and arrive at a solution.

Less structured than other more traditional problem solving methods, the 5 Whys is simply what it sounds like: asking why over and over to get to the root of an obstacle or setback. This technique encourages an open dialogue that can trigger new ideas about a problem, whether done individually or with a group. Each why piggybacks off the answer to the previous why. Get started with the template below—both flowcharts and fishbone diagrams can also help you track your answers to the 5 Whys.

Brainstorming

A meeting of the minds, a brain dump, a mind meld, a jam session. Whatever you call it, collaborative brainstorming can help surface previously unseen issues, root causes, and alternative solutions. Create and share a mind map with your team members to fuel your brainstorming session.

Gap analysis

Sometimes you don’t know where the problem is until you determine where it isn’t. Gap filling helps you analyze inadequacies that are preventing you from reaching an optimized state or end goal. For example, a content gap analysis can help a content marketer determine where holes exist in messaging or the customer experience. Gap analysis is especially helpful when it comes to problem solving because it requires you to find workable solutions. A SWOT analysis chart that looks at a problem through the lens of strengths, opportunities, opportunities, and threats can be a helpful problem solving framework as you start your analysis.

A better way to problem solve

Beyond these practical tips and tools, there are myriad methodical and creative approaches to move a project forward or resolve a conflict. The right approach will depend on the scope of the issue and your desired outcome.

Depending on the problem, Lucidchart offers several templates and diagrams that could help you identify the cause of the issue and map out a plan to resolve it.  Learn more about how Lucidchart can help you take control of your problem solving process .

Lucidchart, a cloud-based intelligent diagramming application, is a core component of Lucid Software's Visual Collaboration Suite. This intuitive, cloud-based solution empowers teams to collaborate in real-time to build flowcharts, mockups, UML diagrams, customer journey maps, and more. Lucidchart propels teams forward to build the future faster. Lucid is proud to serve top businesses around the world, including customers such as Google, GE, and NBC Universal, and 99% of the Fortune 500. Lucid partners with industry leaders, including Google, Atlassian, and Microsoft. Since its founding, Lucid has received numerous awards for its products, business, and workplace culture. For more information, visit lucidchart.com.

Related articles

Sometimes you're faced with challenges that traditional problem solving can't fix. Creative problem solving encourages you to find new, creative ways of thinking that can help you overcome the issue at hand more quickly.

Root cause analysis refers to any problem-solving method used to trace an issue back to its origin. Learn how to complete a root cause analysis—we've even included templates to get you started.

Bring your bright ideas to life.

or continue with

Complex cognition: the science of human reasoning, problem-solving, and decision-making

• Published: 23 March 2010
• Volume 11 , pages 99–102, ( 2010 )

Cite this article

• Markus Knauff 1 &
• Ann G. Wolf 1  

19k Accesses

39 Citations

Explore all metrics

Avoid common mistakes on your manuscript.

Climate change, globalization, policy of peace, and financial market crises—often we are faced with very complex problems. In order to tackle these complex problems, the responsible people should first come to mutual terms. An additional challenge is that typically the involved parties have different (often conflicting) interests and relate the problems to different emotions and wishes. These factors certainly do not ease the quest for a solution to these complex problems.

It is needless to say that the big problems of our time are not easy to solve. Less clear, however, is identifying the causes that led to these problems. Interest conflicts between social groups, the economic and social system or greed—one can think of many responsible factors for the large-scale problems we are currently confronted with.

The present “Special Corner: complex cognition” deals with questions in this regard that have often received little consideration. Under the headline “complex cognition”, we summarize mental activities such as thinking, reasoning, problem - solving, and decision - making that typically rely on the combination and interaction of more elementary processes such as perception, learning, memory, emotion, etc. (cf. Sternberg and Ben-Zeev 2001 ). However, even though complex cognition relies on these elementary functions, the scope of complex cognition research goes beyond the isolated analysis of such elementary mental processes. Two aspects are essential for “complex cognition”: The first aspect refers to the interaction of different mental activities such as perception, memory, learning, reasoning, emotion, etc. The second aspect takes the complexity of the situation into account an agent is confronted with. Based on these two aspects, the term “complex cognition” can be defined in the following way:

Complex psychological processes: We talk about “complex cognition”, when thinking, problem-solving, or decision-making falls back on other cognitive processes such as “perception”, “working memory”, “long-term memory”, “executive processes”, or when the cognitive processes are in close connection with other processes such as “emotion” and “motivation”. The complexity also results from an interaction from a multitude of processes that occur simultaneously or at different points in time and can be realized in different cognitive and/or neuronal structures.

Complex conditions: We also talk about “complex cognition” when the conditions are complex in which a person finds himself and in which conclusions need to be drawn, a problem needs to be solved, or decisions need to be made. The complexity of the conditions or constraints can have different causes. The situation structure itself can be difficult to “see”, or the action alternatives are difficult “to put into effect”. The conditions can themselves comprise of many different variables. These variables can exhibit a high level of interdependence and cross-connection, and it can, as time passes by, come to a change of the original conditions (e.g. Dörner and Wearing 1995 ; Osman 2010 ). It can also be the case that the problem is embedded in a larger social context and can be solved only under certain specifications (norms, data, legislations, culture, etc.) or that the problem can only be solved in interaction with other agents, be it other persons or technical systems.

When one summarizes these two aspects, this yields the following view of what should be understood as “complex cognition”.

As “complex cognition” we define all mental processes that are used by individuals for deriving new information out of given information, with the intention to solve problems, make decision, and plan actions. The crucial characteristic of “complex cognition” is that it takes place under complex conditions in which a multitude of cognitive processes interact with one another or with other noncognitive processes.

The “Special Corner: complex cognition” deals with complex cognition from many different perspectives. The typical questions of all contributions are: Does the design of the human mind enable the necessary thinking skills to solve the truly complex problems we are faced with? Where lay the boundaries of our thinking skills? How do people derive at conclusions? What makes a problem a complex problem? How can we improve our skills to effectively solve problems and make sound judgements?

It is for sure too much to expect that the Special Corner answers these questions. If it were that easy, we would not be still searching for an answer. It is, however, our intention with the current collection of articles to bring to focus such questions to a larger extent than has been done so far.

An important starting point is the fact that people’s skills to solve the most complex of all problems and to ponder about the most complex issues is often immense—humankind would not otherwise be there were she is now. Yet, on the other hand, it has become more clear in the past few years that often people drift away from what one would identify as “rational” (Kahneman 2003 ). People hardly ever adhere to that what the norms of logic, the probability calculus, or the mathematical decision theory state. For example, most people (and organizations) typically accept more losses for a potential high gain than would be the case if they were to take into account the rules of the probability theory. Similarly, they draw conclusions from received information in a way that is not according to the rules of logic. When people, for example, accept the rule “If it rains, then the street is wet”, they most often conclude that when the street is wet, it must have rained. That, however, is incorrect from a logical perspective: perhaps a cleaning car just drove by. In psychology, two main views are traditionally put forward to explain how such deviations from the normative guidelines occur. One scientific stream is interested in how deviations from the normative models can be explained (Evans 2005 ; Johnson-Laird 2008 ; Knauff 2007 ; Reason 1990 ). According to this line of research, deviations are caused by the limitations of the human cognitive system. The other psychological stream puts forward as the main criticism that the deviations can actually be regarded as mistakes (Gigerenzer 2008 ). The deviations accordingly have a high value, because they are adjusted to the information structure of the environment (Gigerenzer et al. 1999 ). They have probably developed during evolution, because they could ensure survival as for example the specifications of formal logic (Hertwig and Herzog 2009 ). We, the editors of the special corner, are very pleased that we can offer an impression of this debate with the contributions from Marewski, Gaissmaier, and Gigerenzer and the commentaries to this contribution from Evans and Over. Added to this is a reply from Marewski, Gaissmaier, and Gigerenzer to the commentary from Evans and Over.

Another topic in the area of complex cognition can be best illustrated by means of the climate protection. To be successful in this area, the responsible actors have to consider a multitude of ecological, biological, geological, political, and economical factors, the basic conditions are constantly at change, and the intervention methods are not clear. Because the necessary information is not readily available for the person dealing with the problem, the person is forced to obtain the relevant information from other sources. Furthermore, intervention in the complex variable structure of the climate can trigger processes whose impact was likely not intended. Finally, the system will not “wait” for intervention of the actors but will change itself over time. The special corner is also concerned with thinking and problem-solving in such complex situations. The article by Funke gives an overview of the current state of research on this topic from the viewpoint of the author, in which several research areas are covered that have internationally not received much acknowledgement (but see, for example, Osman 2010 ).

Although most contributions to the special corner come from the area of psychology, the contribution by Ragni and Löffler illustrates that computer science can provide a valuable addition to the understanding of complex cognition. Computer science plays an important role in complex cognition. In general, computer science, which is used to investigate computational processes central to all research approaches, can be placed in a “computational theory of cognition” framework. This is true especially for the development of computational theories of complex cognitive processes. In many of our modern knowledge domains, the application of simulations and modelling has become a major part of the methods inventory. Simulations help forecast the weather and climate change, help govern traffic flow and help comprehend physical processes. Although modelling in these areas is a vastly established method, it has been very little applied in the area of human thinking (but see e.g. Anderson 1990 ; Gray 2007 ). However, exactly in the area of complex cognition, the method of cognitive modelling offers empirical research an additional methodological access to the description and explanation of complex cognitive processes. While the validity of psychological theories can be tested with the use of empirical research, cognitive models, with their internal coherence, make possible to test consistency and completeness (e.g. Schmid 2008 ). They will also lead to new hypotheses that will in turn be possible to test experimentally. The contribution of Ragni and Löffler demonstrates with the help of an interesting example, finding the optimal route, the usefulness of simulation and modelling in psychology.

A further problem in the area of complex cognition is that many problems are solvable only under certain social conditions (norms, values, laws, culture) or only in interaction with other actors (cf. Beller 2008 ). The article on deontic reasoning by Beller is concerned with this topic. Deontic reasoning is thinking about whether actions are forbidden or allowed, obligatory or not obligatory. Beller proposes that social norms, imposing constraints on individual actions, constitute the fundamental concept for deontic thinking and that people reason from such norms flexibly according to deontic core principles. The review paper shows how knowing what in a certain situation is allowed or forbidden can influence how people derive at conclusions.

The article of Waldmann, Meder, von Sydow, and Hagmayer is concerned with the important topic of causal reasoning. More specifically, the authors explore the interaction between category and causal induction in causal model learning. The paper is a good example of how experimental work in psychology can combine different research traditions that typically work quite isolated. The paper goes beyond a divide and conquers approach and shows that causal knowledge plays an important role in learning, categorization, perception, decision-making, problem-solving, and text comprehension. In each of these fields, separate theories have been developed to investigate the role of causal knowledge. The first author of the paper is internationally well known for his work on the role of causality in other cognitive functions, in particular in categorization and learning (e.g. Lagnado et al. 2007 ; Waldmann et al. 1995 ). In a number of experimental studies, Waldmann and his colleagues have shown that people when learning about causal relations do not simply form associations between causes and effects but make use of abstract prior assumptions about the underlying causal structure and functional form (Waldmann 2007 ).

We, the guest editors, are very pleased that we have the opportunity with this Special corner to make accessible the topic “complex cognition” to the interdisciplinary readership of Cognitive Processing . We predict a bright future for this topic. The research topic possesses high research relevance in the area of basic research for a multitude of disciplines, for example psychology, computer science, and neuroscience. In addition, this area forms a good foundation for an interdisciplinary cooperation.

A further important reason for the positive development of the area is that the relevance of the area goes beyond fundamental research. In that way, the results of the area can for example also contribute to better understanding of the possibilities and borders of human thinking, problem-solving, and decisions in politics, corporations, and economy. In the long term, it might even lead to practical directions on how to avoid “mistakes” and help us better understand the global challenges of our time—Climate change, globalization, financial market crises, etc.

We thank all the authors for their insightful and inspiring contributions, a multitude of reviewers for their help, the editor-in-chief Marta Olivetti Belardinelli that she gave us the opportunity to address this topic, and the editorial manager, Thomas Hünefeldt, for his support for accomplishing the Special Corner. We wish the readers of the Special Corner lots of fun with reading the contributions!

Anderson JR (1990) The adaptive character of thought. Erlbaum, Hillsdale

Google Scholar  

Beller S (2008) Deontic norms, deontic reasoning, and deontic conditionals. Think Reason 14:305–341

Article   Google Scholar  

Dörner D, Wearing A (1995) Complex problem solving: toward a (computer-simulated) theory. In: Frensch PA, Funke J (eds) Complex problem solving: the European perspective. Lawrence Erlbaum Associates, Hillsdale, pp 65–99

Evans JSBT (2005) Deductive reasoning. In: Holyoak KJ, Morrison RG (eds) The Cambridge handbook of thinking and reasoning. Cambridge University Press, Cambridge, pp 169–184

Gigerenzer G (2008) Rationality for mortals: how people cope with uncertainty. Oxford University Press, Oxford

Gigerenzer G, Todd PM, The ABC Research Group (1999) Simple heuristics that make us smart. Oxford University Press, New York

Gray WD (2007) Integrated models of cognitive systems. Oxford University Press, Oxford

Hertwig R, Herzog SM (2009) Fast and frugal heuristics: tools of social rationality. Soc Cogn 27:661–698

Johnson-Laird PN (2008) Mental models and deductive reasoning. In: Rips L, Adler J (eds) Reasoning: studies in human inference and its foundations. Cambridge University Press, Cambridge, pp 206–222

Kahneman D (2003) A perspective on judgment and choice: mapping bounded rationality. Am Psychol 58:697–720

Article   PubMed   Google Scholar  

Knauff M (2007) How our brains reason logically. Topio 26:19–36

Lagnado DA, Waldmann MR, Hagmayer Y, Sloman SA (2007) Beyond covariation: cues to causal structure. In: Gopnik A, Schulz L (eds) Causal learning: psychology, philosophy, and computation. Oxford University Press, Oxford, pp 154–172

Osman M (2010) Controlling uncertainty: a review of human behavior in complex dynamic environments. Psychol Bull 136(1):65–86

Reason J (1990) Human error. Cambridge University Press, Cambridge

Schmid U (2008) Cognition and AI. KI 08/1, Themenheft “Kognition’’, pp 5–7

Sternberg RJ, Ben-Zeev T (2001) Complex cognition: the psychology of human thought. Oxford University Press, New York

Waldmann MR (2007) Combining versus analyzing multiple causes: how domain assumptions and task context affect integration rules. Cogn Sci 31:233–256

Waldmann MR, Holyoak KJ, Fratianne A (1995) Causal models and the acquisition of category structure. J Exp Psychol Gen 124:181–206

Download references

Author information

Authors and affiliations.

University of Giessen, Giessen, Germany

Markus Knauff & Ann G. Wolf

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Markus Knauff .

Rights and permissions

Reprints and permissions

About this article

Knauff, M., Wolf, A.G. Complex cognition: the science of human reasoning, problem-solving, and decision-making. Cogn Process 11 , 99–102 (2010). https://doi.org/10.1007/s10339-010-0362-z

Download citation

Received : 10 March 2010

Accepted : 10 March 2010

Published : 23 March 2010

Issue Date : May 2010

DOI : https://doi.org/10.1007/s10339-010-0362-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

Word Embedding Analysis Website

LSA & word2vec

Word embedding website executive summary.

Welcome lsa.colorado.edu users! This is an updated website that encompasses all of the old lsa.colorado.edu functionality and more.

Semantic analysis of language is commonly performed using high-dimensional vector space word embeddings of text. These embeddings are generated under the premise of distributional semantics, whereby "a word is characterized by the company it keeps" (John R. Firth). Thus, words that appear in similar contexts are semantically related to one another and consequently will be close in distance to one another in a derived embedding space. This approach has served as the basis for a number of widely used word embedding methods.

Approaches to the generation of word embeddings have evolved over the years: an early technique is Latent Semantic Analysis ( Deerwester et al., 1990 , Landauer, Foltz & Laham, 1998 ) and more recently word2vec ( Mikolov et al., 2013 ). LSA performs a singular value decomposition on a sparse word type to document matrix to obtain lower dimensional vectors of each of the types. Word2vec uses a neural network-based word embedding model trained on a large corpus of text to predict either a word given its context (continuous bag of words; CBOW) or the context surrounding a given word (skip-gram). Contemporary examples of word embedding techniques include ELMo, BERT, GPT-3, XLNet.

The analysis tools available on this website harness LSA, word2vec, and BERT word embeddings. Others may be provided later.

Quick Links

Information.

First time user? See the informational page on word embedding analysis for an overview of word embeddings. For information on how to perform word embedding analyses using this website, see the how to page.

See the FAQ page for answers to frequently asked questions.

CONCEPTUAL ANALYSIS article

Complex problem solving: what it is and what it is not.

• 1 Department of Psychology, University of Bamberg, Bamberg, Germany
• 2 Department of Psychology, Heidelberg University, Heidelberg, Germany

Computer-simulated scenarios have been part of psychological research on problem solving for more than 40 years. The shift in emphasis from simple toy problems to complex, more real-life oriented problems has been accompanied by discussions about the best ways to assess the process of solving complex problems. Psychometric issues such as reliable assessments and addressing correlations with other instruments have been in the foreground of these discussions and have left the content validity of complex problem solving in the background. In this paper, we return the focus to content issues and address the important features that define complex problems.

Succeeding in the 21st century requires many competencies, including creativity, life-long learning, and collaboration skills (e.g., National Research Council, 2011 ; Griffin and Care, 2015 ), to name only a few. One competence that seems to be of central importance is the ability to solve complex problems ( Mainzer, 2009 ). Mainzer quotes the Nobel prize winner Simon (1957) who wrote as early as 1957:

The capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problem whose solution is required for objectively rational behavior in the real world or even for a reasonable approximation to such objective rationality. (p. 198)

The shift from well-defined to ill-defined problems came about as a result of a disillusion with the “general problem solver” ( Newell et al., 1959 ): The general problem solver was a computer software intended to solve all kind of problems that can be expressed through well-formed formulas. However, it soon became clear that this procedure was in fact a “special problem solver” that could only solve well-defined problems in a closed space. But real-world problems feature open boundaries and have no well-determined solution. In fact, the world is full of wicked problems and clumsy solutions ( Verweij and Thompson, 2006 ). As a result, solving well-defined problems and solving ill-defined problems requires different cognitive processes ( Schraw et al., 1995 ; but see Funke, 2010 ).

Well-defined problems have a clear set of means for reaching a precisely described goal state. For example: in a match-stick arithmetic problem, a person receives a false arithmetic expression constructed out of matchsticks (e.g., IV = III + III). According to the instructions, moving one of the matchsticks will make the equations true. Here, both the problem (find the appropriate stick to move) and the goal state (true arithmetic expression; solution is: VI = III + III) are defined clearly.

Ill-defined problems have no clear problem definition, their goal state is not defined clearly, and the means of moving towards the (diffusely described) goal state are not clear. For example: The goal state for solving the political conflict in the near-east conflict between Israel and Palestine is not clearly defined (living in peaceful harmony with each other?) and even if the conflict parties would agree on a two-state solution, this goal again leaves many issues unresolved. This type of problem is called a “complex problem” and is of central importance to this paper. All psychological processes that occur within individual persons and deal with the handling of such ill-defined complex problems will be subsumed under the umbrella term “complex problem solving” (CPS).

Systematic research on CPS started in the 1970s with observations of the behavior of participants who were confronted with computer simulated microworlds. For example, in one of those microworlds participants assumed the role of executives who were tasked to manage a company over a certain period of time (see Brehmer and Dörner, 1993 , for a discussion of this methodology). Today, CPS is an established concept and has even influenced large-scale assessments such as PISA (“Programme for International Student Assessment”), organized by the Organization for Economic Cooperation and Development ( OECD, 2014 ). According to the World Economic Forum, CPS is one of the most important competencies required in the future ( World Economic Forum, 2015 ). Numerous articles on the subject have been published in recent years, documenting the increasing research activity relating to this field. In the following collection of papers we list only those published in 2010 and later: theoretical papers ( Blech and Funke, 2010 ; Funke, 2010 ; Knauff and Wolf, 2010 ; Leutner et al., 2012 ; Selten et al., 2012 ; Wüstenberg et al., 2012 ; Greiff et al., 2013b ; Fischer and Neubert, 2015 ; Schoppek and Fischer, 2015 ), papers about measurement issues ( Danner et al., 2011a ; Greiff et al., 2012 , 2015a ; Alison et al., 2013 ; Gobert et al., 2015 ; Greiff and Fischer, 2013 ; Herde et al., 2016 ; Stadler et al., 2016 ), papers about applications ( Fischer and Neubert, 2015 ; Ederer et al., 2016 ; Tremblay et al., 2017 ), papers about differential effects ( Barth and Funke, 2010 ; Danner et al., 2011b ; Beckmann and Goode, 2014 ; Greiff and Neubert, 2014 ; Scherer et al., 2015 ; Meißner et al., 2016 ; Wüstenberg et al., 2016 ), one paper about developmental effects ( Frischkorn et al., 2014 ), one paper with a neuroscience background ( Osman, 2012 ) 1 , papers about cultural differences ( Güss and Dörner, 2011 ; Sonnleitner et al., 2014 ; Güss et al., 2015 ), papers about validity issues ( Goode and Beckmann, 2010 ; Greiff et al., 2013c ; Schweizer et al., 2013 ; Mainert et al., 2015 ; Funke et al., 2017 ; Greiff et al., 2017 , 2015b ; Kretzschmar et al., 2016 ; Kretzschmar, 2017 ), review papers and meta-analyses ( Osman, 2010 ; Stadler et al., 2015 ), and finally books ( Qudrat-Ullah, 2015 ; Csapó and Funke, 2017b ) and book chapters ( Funke, 2012 ; Hotaling et al., 2015 ; Funke and Greiff, 2017 ; Greiff and Funke, 2017 ; Csapó and Funke, 2017a ; Fischer et al., 2017 ; Molnàr et al., 2017 ; Tobinski and Fritz, 2017 ; Viehrig et al., 2017 ). In addition, a new “Journal of Dynamic Decision Making” (JDDM) has been launched ( Fischer et al., 2015 , 2016 ) to give the field an open-access outlet for research and discussion.

This paper aims to clarify aspects of validity: what should be meant by the term CPS and what not? This clarification seems necessary because misunderstandings in recent publications provide – from our point of view – a potentially misleading picture of the construct. We start this article with a historical review before attempting to systematize different positions. We conclude with a working definition.

Historical Review

The concept behind CPS goes back to the German phrase “komplexes Problemlösen” (CPS; the term “komplexes Problemlösen” was used as a book title by Funke, 1986 ). The concept was introduced in Germany by Dörner and colleagues in the mid-1970s (see Dörner et al., 1975 ; Dörner, 1975 ) for the first time. The German phrase was later translated to CPS in the titles of two edited volumes by Sternberg and Frensch (1991) and Frensch and Funke (1995a) that collected papers from different research traditions. Even though it looks as though the term was coined in the 1970s, Edwards (1962) used the term “dynamic decision making” to describe decisions that come in a sequence. He compared static with dynamic decision making, writing:

In dynamic situations, a new complication not found in the static situations arises. The environment in which the decision is set may be changing, either as a function of the sequence of decisions, or independently of them, or both. It is this possibility of an environment which changes while you collect information about it which makes the task of dynamic decision theory so difficult and so much fun. (p. 60)

The ability to solve complex problems is typically measured via dynamic systems that contain several interrelated variables that participants need to alter. Early work (see, e.g., Dörner, 1980 ) used a simulation scenario called “Lohhausen” that contained more than 2000 variables that represented the activities of a small town: Participants had to take over the role of a mayor for a simulated period of 10 years. The simulation condensed these ten years to ten hours in real time. Later, researchers used smaller dynamic systems as scenarios either based on linear equations (see, e.g., Funke, 1993 ) or on finite state automata (see, e.g., Buchner and Funke, 1993 ). In these contexts, CPS consisted of the identification and control of dynamic task environments that were previously unknown to the participants. Different task environments came along with different degrees of fidelity ( Gray, 2002 ).

According to Funke (2012) , the typical attributes of complex systems are (a) complexity of the problem situation which is usually represented by the sheer number of involved variables; (b) connectivity and mutual dependencies between involved variables; (c) dynamics of the situation, which reflects the role of time and developments within a system; (d) intransparency (in part or full) about the involved variables and their current values; and (e) polytely (greek term for “many goals”), representing goal conflicts on different levels of analysis. This mixture of features is similar to what is called VUCA (volatility, uncertainty, complexity, ambiguity) in modern approaches to management (e.g., Mack et al., 2016 ).

In his evaluation of the CPS movement, Sternberg (1995) compared (young) European approaches to CPS with (older) American research on expertise. His analysis of the differences between the European and American traditions shows advantages but also potential drawbacks for each side. He states (p. 301): “I believe that although there are problems with the European approach, it deals with some fundamental questions that American research scarcely addresses.” So, even though the echo of the European approach did not enjoy strong resonance in the US at that time, it was valued by scholars like Sternberg and others. Before attending to validity issues, we will first present a short review of different streams.

Different Approaches to CPS

In the short history of CPS research, different approaches can be identified ( Buchner, 1995 ; Fischer et al., 2017 ). To systematize, we differentiate between the following five lines of research:

(a) The search for individual differences comprises studies identifying interindividual differences that affect the ability to solve complex problems. This line of research is reflected, for example, in the early work by Dörner et al. (1983) and their “Lohhausen” study. Here, naïve student participants took over the role of the mayor of a small simulated town named Lohhausen for a simulation period of ten years. According to the results of the authors, it is not intelligence (as measured by conventional IQ tests) that predicts performance, but it is the ability to stay calm in the face of a challenging situation and the ability to switch easily between an analytic mode of processing and a more holistic one.

(b) The search for cognitive processes deals with the processes behind understanding complex dynamic systems. Representative of this line of research is, for example, Berry and Broadbent’s (1984) work on implicit and explicit learning processes when people interact with a dynamic system called “Sugar Production”. They found that those who perform best in controlling a dynamic system can do so implicitly, without explicit knowledge of details regarding the systems’ relations.

(c) The search for system factors seeks to identify the aspects of dynamic systems that determine the difficulty of complex problems and make some problems harder than others. Representative of this line of research is, for example, work by Funke (1985) , who systematically varied the number of causal effects within a dynamic system or the presence/absence of eigendynamics. He found, for example, that solution quality decreases as the number of systems relations increases.

(d) The psychometric approach develops measurement instruments that can be used as an alternative to classical IQ tests, as something that goes “beyond IQ”. The MicroDYN approach ( Wüstenberg et al., 2012 ) is representative for this line of research that presents an alternative to reasoning tests (like Raven matrices). These authors demonstrated that a small improvement in predicting school grade point average beyond reasoning is possible with MicroDYN tests.

(e) The experimental approach explores CPS under different experimental conditions. This approach uses CPS assessment instruments to test hypotheses derived from psychological theories and is sometimes used in research about cognitive processes (see above). Exemplary for this line of research is the work by Rohe et al. (2016) , who test the usefulness of “motto goals” in the context of complex problems compared to more traditional learning and performance goals. Motto goals differ from pure performance goals by activating positive affect and should lead to better goal attainment especially in complex situations (the mentioned study found no effect).

To be clear: these five approaches are not mutually exclusive and do overlap. But the differentiation helps to identify different research communities and different traditions. These communities had different opinions about scaling complexity.

The Race for Complexity: Use of More and More Complex Systems

In the early years of CPS research, microworlds started with systems containing about 20 variables (“Tailorshop”), soon reached 60 variables (“Moro”), and culminated in systems with about 2000 variables (“Lohhausen”). This race for complexity ended with the introduction of the concept of “minimal complex systems” (MCS; Greiff and Funke, 2009 ; Funke and Greiff, 2017 ), which ushered in a search for the lower bound of complexity instead of the higher bound, which could not be defined as easily. The idea behind this concept was that whereas the upper limits of complexity are unbound, the lower limits might be identifiable. Imagine starting with a simple system containing two variables with a simple linear connection between them; then, step by step, increase the number of variables and/or the type of connections. One soon reaches a point where the system can no longer be considered simple and has become a “complex system”. This point represents a minimal complex system. Despite some research having been conducted in this direction, the point of transition from simple to complex has not been identified clearly as of yet.

Some years later, the original “minimal complex systems” approach ( Greiff and Funke, 2009 ) shifted to the “multiple complex systems” approach ( Greiff et al., 2013a ). This shift is more than a slight change in wording: it is important because it taps into the issue of validity directly. Minimal complex systems have been introduced in the context of challenges from large-scale assessments like PISA 2012 that measure new aspects of problem solving, namely interactive problems besides static problem solving ( Greiff and Funke, 2017 ). PISA 2012 required test developers to remain within testing time constraints (given by the school class schedule). Also, test developers needed a large item pool for the construction of a broad class of problem solving items. It was clear from the beginning that MCS deal with simple dynamic situations that require controlled interaction: the exploration and control of simple ticket machines, simple mobile phones, or simple MP3 players (all of these example domains were developed within PISA 2012) – rather than really complex situations like managerial or political decision making.

As a consequence of this subtle but important shift in interpreting the letters MCS, the definition of CPS became a subject of debate recently ( Funke, 2014a ; Greiff and Martin, 2014 ; Funke et al., 2017 ). In the words of Funke (2014b , p. 495):

It is funny that problems that nowadays come under the term ‘CPS’, are less complex (in terms of the previously described attributes of complex situations) than at the beginning of this new research tradition. The emphasis on psychometric qualities has led to a loss of variety. Systems thinking requires more than analyzing models with two or three linear equations – nonlinearity, cyclicity, rebound effects, etc. are inherent features of complex problems and should show up at least in some of the problems used for research and assessment purposes. Minimal complex systems run the danger of becoming minimal valid systems.

Searching for minimal complex systems is not the same as gaining insight into the way how humans deal with complexity and uncertainty. For psychometric purposes, it is appropriate to reduce complexity to a minimum; for understanding problem solving under conditions of overload, intransparency, and dynamics, it is necessary to realize those attributes with reasonable strength. This aspect is illustrated in the next section.

Importance of the Validity Issue

The most important reason for discussing the question of what complex problem solving is and what it is not stems from its phenomenology: if we lose sight of our phenomena, we are no longer doing good psychology. The relevant phenomena in the context of complex problems encompass many important aspects. In this section, we discuss four phenomena that are specific to complex problems. We consider these phenomena as critical for theory development and for the construction of assessment instruments (i.e., microworlds). These phenomena require theories for explaining them and they require assessment instruments eliciting them in a reliable way.

The first phenomenon is the emergency reaction of the intellectual system ( Dörner, 1980 ): When dealing with complex systems, actors tend to (a) reduce their intellectual level by decreasing self-reflections, by decreasing their intentions, by stereotyping, and by reducing their realization of intentions, (b) they show a tendency for fast action with increased readiness for risk, with increased violations of rules, and with increased tendency to escape the situation, and (c) they degenerate their hypotheses formation by construction of more global hypotheses and reduced tests of hypotheses, by increasing entrenchment, and by decontextualizing their goals. This phenomenon illustrates the strong connection between cognition, emotion, and motivation that has been emphasized by Dörner (see, e.g., Dörner and Güss, 2013 ) from the beginning of his research tradition; the emergency reaction reveals a shift in the mode of information processing under the pressure of complexity.

The second phenomenon comprises cross-cultural differences with respect to strategy use ( Strohschneider and Güss, 1999 ; Güss and Wiley, 2007 ; Güss et al., 2015 ). Results from complex task environments illustrate the strong influence of context and background knowledge to an extent that cannot be found for knowledge-poor problems. For example, in a comparison between Brazilian and German participants, it turned out that Brazilians accept the given problem descriptions and are more optimistic about the results of their efforts, whereas Germans tend to inquire more about the background of the problems and take a more active approach but are less optimistic (according to Strohschneider and Güss, 1998 , p. 695).

The third phenomenon relates to failures that occur during the planning and acting stages ( Jansson, 1994 ; Ramnarayan et al., 1997 ), illustrating that rational procedures seem to be unlikely to be used in complex situations. The potential for failures ( Dörner, 1996 ) rises with the complexity of the problem. Jansson (1994) presents seven major areas for failures with complex situations: acting directly on current feedback; insufficient systematization; insufficient control of hypotheses and strategies; lack of self-reflection; selective information gathering; selective decision making; and thematic vagabonding.

The fourth phenomenon describes (a lack of) training and transfer effects ( Kretzschmar and Süß, 2015 ), which again illustrates the context dependency of strategies and knowledge (i.e., there is no strategy that is so universal that it can be used in many different problem situations). In their own experiment, the authors could show training effects only for knowledge acquisition, not for knowledge application. Only with specific feedback, performance in complex environments can be increased ( Engelhart et al., 2017 ).

These four phenomena illustrate why the type of complexity (or degree of simplicity) used in research really matters. Furthermore, they demonstrate effects that are specific for complex problems, but not for toy problems. These phenomena direct the attention to the important question: does the stimulus material used (i.e., the computer-simulated microworld) tap and elicit the manifold of phenomena described above?

Dealing with partly unknown complex systems requires courage, wisdom, knowledge, grit, and creativity. In creativity research, “little c” and “BIG C” are used to differentiate between everyday creativity and eminent creativity ( Beghetto and Kaufman, 2007 ; Kaufman and Beghetto, 2009 ). Everyday creativity is important for solving everyday problems (e.g., finding a clever fix for a broken spoke on my bicycle), eminent creativity changes the world (e.g., inventing solar cells for energy production). Maybe problem solving research should use a similar differentiation between “little p” and “BIG P” to mark toy problems on the one side and big societal challenges on the other. The question then remains: what can we learn about BIG P by studying little p? What phenomena are present in both types, and what phenomena are unique to each of the two extremes?

Discussing research on CPS requires reflecting on the field’s research methods. Even if the experimental approach has been successful for testing hypotheses (for an overview of older work, see Funke, 1995 ), other methods might provide additional and novel insights. Complex phenomena require complex approaches to understand them. The complex nature of complex systems imposes limitations on psychological experiments: The more complex the environments, the more difficult is it to keep conditions under experimental control. And if experiments have to be run in labs one should bring enough complexity into the lab to establish the phenomena mentioned, at least in part.

There are interesting options to be explored (again): think-aloud protocols , which have been discredited for many years ( Nisbett and Wilson, 1977 ) and yet are a valuable source for theory testing ( Ericsson and Simon, 1983 ); introspection ( Jäkel and Schreiber, 2013 ), which seems to be banned from psychological methods but nevertheless offers insights into thought processes; the use of life-streaming ( Wendt, 2017 ), a medium in which streamers generate a video stream of think-aloud data in computer-gaming; political decision-making ( Dhami et al., 2015 ) that demonstrates error-proneness in groups; historical case studies ( Dörner and Güss, 2011 ) that give insights into the thinking styles of political leaders; the use of the critical incident technique ( Reuschenbach, 2008 ) to construct complex scenarios; and simulations with different degrees of fidelity ( Gray, 2002 ).

The methods tool box is full of instruments that have to be explored more carefully before any individual instrument receives a ban or research narrows its focus to only one paradigm for data collection. Brehmer and Dörner (1993) discussed the tensions between “research in the laboratory and research in the field”, optimistically concluding “that the new methodology of computer-simulated microworlds will provide us with the means to bridge the gap between the laboratory and the field” (p. 183). The idea behind this optimism was that computer-simulated scenarios would bring more complexity from the outside world into the controlled lab environment. But this is not true for all simulated scenarios. In his paper on simulated environments, Gray (2002) differentiated computer-simulated environments with respect to three dimensions: (1) tractability (“the more training subjects require before they can use a simulated task environment, the less tractable it is”, p. 211), correspondence (“High correspondence simulated task environments simulate many aspects of one task environment. Low correspondence simulated task environments simulate one aspect of many task environments”, p. 214), and engagement (“A simulated task environment is engaging to the degree to which it involves and occupies the participants; that is, the degree to which they agree to take it seriously”, p. 217). But the mere fact that a task is called a “computer-simulated task environment” does not mean anything specific in terms of these three dimensions. This is one of several reasons why we should differentiate between those studies that do not address the core features of CPS and those that do.

What is not CPS?

Even though a growing number of references claiming to deal with complex problems exist (e.g., Greiff and Wüstenberg, 2015 ; Greiff et al., 2016 ), it would be better to label the requirements within these tasks “dynamic problem solving,” as it has been done adequately in earlier work ( Greiff et al., 2012 ). The dynamics behind on-off-switches ( Thimbleby, 2007 ) are remarkable but not really complex. Small nonlinear systems that exhibit stunningly complex and unstable behavior do exist – but they are not used in psychometric assessments of so-called CPS. There are other small systems (like MicroDYN scenarios: Greiff and Wüstenberg, 2014 ) that exhibit simple forms of system behavior that are completely predictable and stable. This type of simple systems is used frequently. It is even offered commercially as a complex problem-solving test called COMPRO ( Greiff and Wüstenberg, 2015 ) for business applications. But a closer look reveals that the label is not used correctly; within COMPRO, the used linear equations are far from being complex and the system can be handled properly by using only one strategy (see for more details Funke et al., 2017 ).

Why do simple linear systems not fall within CPS? At the surface, nonlinear and linear systems might appear similar because both only include 3–5 variables. But the difference is in terms of systems behavior as well as strategies and learning. If the behavior is simple (as in linear systems where more input is related to more output and vice versa), the system can be easily understood (participants in the MicroDYN world have 3 minutes to explore a complex system). If the behavior is complex (as in systems that contain strange attractors or negative feedback loops), things become more complicated and much more observation is needed to identify the hidden structure of the unknown system ( Berry and Broadbent, 1984 ; Hundertmark et al., 2015 ).

Another issue is learning. If tasks can be solved using a single (and not so complicated) strategy, steep learning curves are to be expected. The shift from problem solving to learned routine behavior occurs rapidly, as was demonstrated by Luchins (1942) . In his water jar experiments, participants quickly acquired a specific strategy (a mental set) for solving certain measurement problems that they later continued applying to problems that would have allowed for easier approaches. In the case of complex systems, learning can occur only on very general, abstract levels because it is difficult for human observers to make specific predictions. Routines dealing with complex systems are quite different from routines relating to linear systems.

What should not be studied under the label of CPS are pure learning effects, multiple-cue probability learning, or tasks that can be solved using a single strategy. This last issue is a problem for MicroDYN tasks that rely strongly on the VOTAT strategy (“vary one thing at a time”; see Tschirgi, 1980 ). In real-life, it is hard to imagine a business manager trying to solve her or his problems by means of VOTAT.

What is CPS?

In the early days of CPS research, planet Earth’s dynamics and complexities gained attention through such books as “The limits to growth” ( Meadows et al., 1972 ) and “Beyond the limits” ( Meadows et al., 1992 ). In the current decade, for example, the World Economic Forum (2016) attempts to identify the complexities and risks of our modern world. In order to understand the meaning of complexity and uncertainty, taking a look at the worlds’ most pressing issues is helpful. Searching for strategies to cope with these problems is a difficult task: surely there is no place for the simple principle of “vary-one-thing-at-a-time” (VOTAT) when it comes to global problems. The VOTAT strategy is helpful in the context of simple problems ( Wüstenberg et al., 2014 ); therefore, whether or not VOTAT is helpful in a given problem situation helps us distinguish simple from complex problems.

Because there exist no clear-cut strategies for complex problems, typical failures occur when dealing with uncertainty ( Dörner, 1996 ; Güss et al., 2015 ). Ramnarayan et al. (1997) put together a list of generic errors (e.g., not developing adequate action plans; lack of background control; learning from experience blocked by stereotype knowledge; reactive instead of proactive action) that are typical of knowledge-rich complex systems but cannot be found in simple problems.

Complex problem solving is not a one-dimensional, low-level construct. On the contrary, CPS is a multi-dimensional bundle of competencies existing at a high level of abstraction, similar to intelligence (but going beyond IQ). As Funke et al. (2018) state: “Assessment of transversal (in educational contexts: cross-curricular) competencies cannot be done with one or two types of assessment. The plurality of skills and competencies requires a plurality of assessment instruments.”

There are at least three different aspects of complex systems that are part of our understanding of a complex system: (1) a complex system can be described at different levels of abstraction; (2) a complex system develops over time, has a history, a current state, and a (potentially unpredictable) future; (3) a complex system is knowledge-rich and activates a large semantic network, together with a broad list of potential strategies (domain-specific as well as domain-general).

Complex problem solving is not only a cognitive process but is also an emotional one ( Spering et al., 2005 ; Barth and Funke, 2010 ) and strongly dependent on motivation (low-stakes versus high-stakes testing; see Hermes and Stelling, 2016 ).

Furthermore, CPS is a dynamic process unfolding over time, with different phases and with more differentiation than simply knowledge acquisition and knowledge application. Ideally, the process should entail identifying problems (see Dillon, 1982 ; Lee and Cho, 2007 ), even if in experimental settings, problems are provided to participants a priori . The more complex and open a given situation, the more options can be generated (T. S. Schweizer et al., 2016 ). In closed problems, these processes do not occur in the same way.

In analogy to the difference between formative (process-oriented) and summative (result-oriented) assessment ( Wiliam and Black, 1996 ; Bennett, 2011 ), CPS should not be reduced to the mere outcome of a solution process. The process leading up to the solution, including detours and errors made along the way, might provide a more differentiated impression of a person’s problem-solving abilities and competencies than the final result of such a process. This is one of the reasons why CPS environments are not, in fact, complex intelligence tests: research on CPS is not only about the outcome of the decision process, but it is also about the problem-solving process itself.

Complex problem solving is part of our daily life: finding the right person to share one’s life with, choosing a career that not only makes money, but that also makes us happy. Of course, CPS is not restricted to personal problems – life on Earth gives us many hard nuts to crack: climate change, population growth, the threat of war, the use and distribution of natural resources. In sum, many societal challenges can be seen as complex problems. To reduce that complexity to a one-hour lab activity on a random Friday afternoon puts it out of context and does not address CPS issues.

Theories about CPS should specify which populations they apply to. Across populations, one thing to consider is prior knowledge. CPS research with experts (e.g., Dew et al., 2009 ) is quite different from problem solving research using tasks that intentionally do not require any specific prior knowledge (see, e.g., Beckmann and Goode, 2014 ).

More than 20 years ago, Frensch and Funke (1995b) defined CPS as follows:

CPS occurs to overcome barriers between a given state and a desired goal state by means of behavioral and/or cognitive, multi-step activities. The given state, goal state, and barriers between given state and goal state are complex, change dynamically during problem solving, and are intransparent. The exact properties of the given state, goal state, and barriers are unknown to the solver at the outset. CPS implies the efficient interaction between a solver and the situational requirements of the task, and involves a solver’s cognitive, emotional, personal, and social abilities and knowledge. (p. 18)

The above definition is rather formal and does not account for content or relations between the simulation and the real world. In a sense, we need a new definition of CPS that addresses these issues. Based on our previous arguments, we propose the following working definition:

Complex problem solving is a collection of self-regulated psychological processes and activities necessary in dynamic environments to achieve ill-defined goals that cannot be reached by routine actions. Creative combinations of knowledge and a broad set of strategies are needed. Solutions are often more bricolage than perfect or optimal. The problem-solving process combines cognitive, emotional, and motivational aspects, particularly in high-stakes situations. Complex problems usually involve knowledge-rich requirements and collaboration among different persons.

The main differences to the older definition lie in the emphasis on (a) the self-regulation of processes, (b) creativity (as opposed to routine behavior), (c) the bricolage type of solution, and (d) the role of high-stakes challenges. Our new definition incorporates some aspects that have been discussed in this review but were not reflected in the 1995 definition, which focused on attributes of complex problems like dynamics or intransparency.

This leads us to the final reflection about the role of CPS for dealing with uncertainty and complexity in real life. We will distinguish thinking from reasoning and introduce the sense of possibility as an important aspect of validity.

CPS as Combining Reasoning and Thinking in an Uncertain Reality

Leading up to the Battle of Borodino in Leo Tolstoy’s novel “War and Peace”, Prince Andrei Bolkonsky explains the concept of war to his friend Pierre. Pierre expects war to resemble a game of chess: You position the troops and attempt to defeat your opponent by moving them in different directions.

“Far from it!”, Andrei responds. “In chess, you know the knight and his moves, you know the pawn and his combat strength. While in war, a battalion is sometimes stronger than a division and sometimes weaker than a company; it all depends on circumstances that can never be known. In war, you do not know the position of your enemy; some things you might be able to observe, some things you have to divine (but that depends on your ability to do so!) and many things cannot even be guessed at. In chess, you can see all of your opponent’s possible moves. In war, that is impossible. If you decide to attack, you cannot know whether the necessary conditions are met for you to succeed. Many a time, you cannot even know whether your troops will follow your orders…”

In essence, war is characterized by a high degree of uncertainty. A good commander (or politician) can add to that what he or she sees, tentatively fill in the blanks – and not just by means of logical deduction but also by intelligently bridging missing links. A bad commander extrapolates from what he sees and thus arrives at improper conclusions.

Many languages differentiate between two modes of mentalizing; for instance, the English language distinguishes between ‘thinking’ and ‘reasoning’. Reasoning denotes acute and exact mentalizing involving logical deductions. Such deductions are usually based on evidence and counterevidence. Thinking, however, is what is required to write novels. It is the construction of an initially unknown reality. But it is not a pipe dream, an unfounded process of fabrication. Rather, thinking asks us to imagine reality (“Wirklichkeitsfantasie”). In other words, a novelist has to possess a “sense of possibility” (“Möglichkeitssinn”, Robert Musil; in German, sense of possibility is often used synonymously with imagination even though imagination is not the same as sense of possibility, for imagination also encapsulates the impossible). This sense of possibility entails knowing the whole (or several wholes) or being able to construe an unknown whole that could accommodate a known part. The whole has to align with sociological and geographical givens, with the mentality of certain peoples or groups, and with the laws of physics and chemistry. Otherwise, the entire venture is ill-founded. A sense of possibility does not aim for the moon but imagines something that might be possible but has not been considered possible or even potentially possible so far.

Thinking is a means to eliminate uncertainty. This process requires both of the modes of thinking we have discussed thus far. Economic, political, or ecological decisions require us to first consider the situation at hand. Though certain situational aspects can be known, but many cannot. In fact, von Clausewitz (1832) posits that only about 25% of the necessary information is available when a military decision needs to be made. Even then, there is no way to guarantee that whatever information is available is also correct: Even if a piece of information was completely accurate yesterday, it might no longer apply today.

Once our sense of possibility has helped grasping a situation, problem solvers need to call on their reasoning skills. Not every situation requires the same action, and we may want to act this way or another to reach this or that goal. This appears logical, but it is a logic based on constantly shifting grounds: We cannot know whether necessary conditions are met, sometimes the assumptions we have made later turn out to be incorrect, and sometimes we have to revise our assumptions or make completely new ones. It is necessary to constantly switch between our sense of possibility and our sense of reality, that is, to switch between thinking and reasoning. It is an arduous process, and some people handle it well, while others do not.

If we are to believe Tuchman’s (1984) book, “The March of Folly”, most politicians and commanders are fools. According to Tuchman, not much has changed in the 3300 years that have elapsed since the misguided Trojans decided to welcome the left-behind wooden horse into their city that would end up dismantling Troy’s defensive walls. The Trojans, too, had been warned, but decided not to heed the warning. Although Laocoön had revealed the horse’s true nature to them by attacking it with a spear, making the weapons inside the horse ring, the Trojans refused to see the forest for the trees. They did not want to listen, they wanted the war to be over, and this desire ended up shaping their perception.

The objective of psychology is to predict and explain human actions and behavior as accurately as possible. However, thinking cannot be investigated by limiting its study to neatly confined fractions of reality such as the realms of propositional logic, chess, Go tasks, the Tower of Hanoi, and so forth. Within these systems, there is little need for a sense of possibility. But a sense of possibility – the ability to divine and construe an unknown reality – is at least as important as logical reasoning skills. Not researching the sense of possibility limits the validity of psychological research. All economic and political decision making draws upon this sense of possibility. By not exploring it, psychological research dedicated to the study of thinking cannot further the understanding of politicians’ competence and the reasons that underlie political mistakes. Christopher Clark identifies European diplomats’, politicians’, and commanders’ inability to form an accurate representation of reality as a reason for the outbreak of World War I. According to Clark’s (2012) book, “The Sleepwalkers”, the politicians of the time lived in their own make-believe world, wrongfully assuming that it was the same world everyone else inhabited. If CPS research wants to make significant contributions to the world, it has to acknowledge complexity and uncertainty as important aspects of it.

For more than 40 years, CPS has been a new subject of psychological research. During this time period, the initial emphasis on analyzing how humans deal with complex, dynamic, and uncertain situations has been lost. What is subsumed under the heading of CPS in modern research has lost the original complexities of real-life problems. From our point of view, the challenges of the 21st century require a return to the origins of this research tradition. We would encourage researchers in the field of problem solving to come back to the original ideas. There is enough complexity and uncertainty in the world to be studied. Improving our understanding of how humans deal with these global and pressing problems would be a worthwhile enterprise.

Author Contributions

JF drafted a first version of the manuscript, DD added further text and commented on the draft. JF finalized the manuscript.

Authors Note

After more than 40 years of controversial discussions between both authors, this is the first joint paper. We are happy to have done this now! We have found common ground!

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors thank the Deutsche Forschungsgemeinschaft (DFG) for the continuous support of their research over many years. Thanks to Daniel Holt for his comments on validity issues, thanks to Julia Nolte who helped us by translating German text excerpts into readable English and helped us, together with Keri Hartman, to improve our style and grammar – thanks for that! We also thank the two reviewers for their helpful critical comments on earlier versions of this manuscript. Finally, we acknowledge financial support by Deutsche Forschungsgemeinschaft and Ruprecht-Karls-Universität Heidelberg within their funding programme Open Access Publishing .

• ^ The fMRI-paper from Anderson (2012) uses the term “complex problem solving” for tasks that do not fall in our understanding of CPS and is therefore excluded from this list.

Alison, L., van den Heuvel, C., Waring, S., Power, N., Long, A., O’Hara, T., et al. (2013). Immersive simulated learning environments for researching critical incidents: a knowledge synthesis of the literature and experiences of studying high-risk strategic decision making. J. Cogn. Eng. Deci. Mak. 7, 255–272. doi: 10.1177/1555343412468113

CrossRef Full Text | Google Scholar

Anderson, J. R. (2012). Tracking problem solving by multivariate pattern analysis and hidden markov model algorithms. Neuropsychologia 50, 487–498. doi: 10.1016/j.neuropsychologia.2011.07.025

PubMed Abstract | CrossRef Full Text | Google Scholar

Barth, C. M., and Funke, J. (2010). Negative affective environments improve complex solving performance. Cogn. Emot. 24, 1259–1268. doi: 10.1080/02699930903223766

Beckmann, J. F., and Goode, N. (2014). The benefit of being naïve and knowing it: the unfavourable impact of perceived context familiarity on learning in complex problem solving tasks. Instruct. Sci. 42, 271–290. doi: 10.1007/s11251-013-9280-7

Beghetto, R. A., and Kaufman, J. C. (2007). Toward a broader conception of creativity: a case for “mini-c” creativity. Psychol. Aesthetics Creat. Arts 1, 73–79. doi: 10.1037/1931-3896.1.2.73

Bennett, R. E. (2011). Formative assessment: a critical review. Assess. Educ. Princ. Policy Pract. 18, 5–25. doi: 10.1080/0969594X.2010.513678

Berry, D. C., and Broadbent, D. E. (1984). On the relationship between task performance and associated verbalizable knowledge. Q. J. Exp. Psychol. 36, 209–231. doi: 10.1080/14640748408402156

Blech, C., and Funke, J. (2010). You cannot have your cake and eat it, too: how induced goal conflicts affect complex problem solving. Open Psychol. J. 3, 42–53. doi: 10.2174/1874350101003010042

Brehmer, B., and Dörner, D. (1993). Experiments with computer-simulated microworlds: escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Comput. Hum. Behav. 9, 171–184. doi: 10.1016/0747-5632(93)90005-D

Buchner, A. (1995). “Basic topics and approaches to the study of complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 27–63.

Google Scholar

Buchner, A., and Funke, J. (1993). Finite state automata: dynamic task environments in problem solving research. Q. J. Exp. Psychol. 46A, 83–118. doi: 10.1080/14640749308401068

Clark, C. (2012). The Sleepwalkers: How Europe Went to War in 1914 . London: Allen Lane.

Csapó, B., and Funke, J. (2017a). “The development and assessment of problem solving in 21st-century schools,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 19–31.

Csapó, B., and Funke, J. (eds) (2017b). The Nature of Problem Solving. Using Research to Inspire 21st Century Learning. Paris: OECD Publishing.

Danner, D., Hagemann, D., Holt, D. V., Hager, M., Schankin, A., Wüstenberg, S., et al. (2011a). Measuring performance in dynamic decision making. Reliability and validity of the Tailorshop simulation. J. Ind. Differ. 32, 225–233. doi: 10.1027/1614-0001/a000055

CrossRef Full Text

Danner, D., Hagemann, D., Schankin, A., Hager, M., and Funke, J. (2011b). Beyond IQ: a latent state-trait analysis of general intelligence, dynamic decision making, and implicit learning. Intelligence 39, 323–334. doi: 10.1016/j.intell.2011.06.004

Dew, N., Read, S., Sarasvathy, S. D., and Wiltbank, R. (2009). Effectual versus predictive logics in entrepreneurial decision-making: differences between experts and novices. J. Bus. Ventur. 24, 287–309. doi: 10.1016/j.jbusvent.2008.02.002

Dhami, M. K., Mandel, D. R., Mellers, B. A., and Tetlock, P. E. (2015). Improving intelligence analysis with decision science. Perspect. Psychol. Sci. 10, 753–757. doi: 10.1177/1745691615598511

Dillon, J. T. (1982). Problem finding and solving. J. Creat. Behav. 16, 97–111. doi: 10.1002/j.2162-6057.1982.tb00326.x

Dörner, D. (1975). Wie Menschen eine Welt verbessern wollten [How people wanted to improve a world]. Bild Der Wissenschaft 12, 48–53.

Dörner, D. (1980). On the difficulties people have in dealing with complexity. Simulat. Gam. 11, 87–106. doi: 10.1177/104687818001100108

Dörner, D. (1996). The Logic of Failure: Recognizing and Avoiding Error in Complex Situations. New York, NY: Basic Books.

Dörner, D., Drewes, U., and Reither, F. (1975). “Über das Problemlösen in sehr komplexen Realitätsbereichen,” in Bericht über den 29. Kongreß der DGfPs in Salzburg 1974, Band 1 , ed. W. H. Tack (Göttingen: Hogrefe), 339–340.

Dörner, D., and Güss, C. D. (2011). A psychological analysis of Adolf Hitler’s decision making as commander in chief: summa confidentia et nimius metus. Rev. Gen. Psychol. 15, 37–49. doi: 10.1037/a0022375

Dörner, D., and Güss, C. D. (2013). PSI: a computational architecture of cognition, motivation, and emotion. Rev. Gen. Psychol. 17, 297–317. doi: 10.1037/a0032947

Dörner, D., Kreuzig, H. W., Reither, F., and Stäudel, T. (1983). Lohhausen. Vom Umgang mit Unbestimmtheit und Komplexität. Bern: Huber.

Ederer, P., Patt, A., and Greiff, S. (2016). Complex problem-solving skills and innovativeness – evidence from occupational testing and regional data. Eur. J. Educ. 51, 244–256. doi: 10.1111/ejed.12176

Edwards, W. (1962). Dynamic decision theory and probabiIistic information processing. Hum. Factors 4, 59–73. doi: 10.1177/001872086200400201

Engelhart, M., Funke, J., and Sager, S. (2017). A web-based feedback study on optimization-based training and analysis of human decision making. J. Dynamic Dec. Mak. 3, 1–23.

Ericsson, K. A., and Simon, H. A. (1983). Protocol Analysis: Verbal Reports As Data. Cambridge, MA: Bradford.

Fischer, A., Greiff, S., and Funke, J. (2017). “The history of complex problem solving,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 107–121.

Fischer, A., Holt, D. V., and Funke, J. (2015). Promoting the growing field of dynamic decision making. J. Dynamic Decis. Mak. 1, 1–3. doi: 10.11588/jddm.2015.1.23807

Fischer, A., Holt, D. V., and Funke, J. (2016). The first year of the “journal of dynamic decision making.” J. Dynamic Decis. Mak. 2, 1–2. doi: 10.11588/jddm.2016.1.28995

Fischer, A., and Neubert, J. C. (2015). The multiple faces of complex problems: a model of problem solving competency and its implications for training and assessment. J. Dynamic Decis. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.23945

Frensch, P. A., and Funke, J. (eds) (1995a). Complex Problem Solving: The European Perspective. Hillsdale, NJ: Erlbaum.

Frensch, P. A., and Funke, J. (1995b). “Definitions, traditions, and a general framework for understanding complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Lawrence Erlbaum), 3–25.

Frischkorn, G. T., Greiff, S., and Wüstenberg, S. (2014). The development of complex problem solving in adolescence: a latent growth curve analysis. J. Educ. Psychol. 106, 1004–1020. doi: 10.1037/a0037114

Funke, J. (1985). Steuerung dynamischer Systeme durch Aufbau und Anwendung subjektiver Kausalmodelle. Z. Psychol. 193, 435–457.

Funke, J. (1986). Komplexes Problemlösen - Bestandsaufnahme und Perspektiven [Complex Problem Solving: Survey and Perspectives]. Heidelberg: Springer.

Funke, J. (1993). “Microworlds based on linear equation systems: a new approach to complex problem solving and experimental results,” in The Cognitive Psychology of Knowledge , eds G. Strube and K.-F. Wender (Amsterdam: Elsevier Science Publishers), 313–330.

Funke, J. (1995). “Experimental research on complex problem solving,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 243–268.

Funke, J. (2010). Complex problem solving: a case for complex cognition? Cogn. Process. 11, 133–142. doi: 10.1007/s10339-009-0345-0

Funke, J. (2012). “Complex problem solving,” in Encyclopedia of the Sciences of Learning , Vol. 38, ed. N. M. Seel (Heidelberg: Springer), 682–685.

Funke, J. (2014a). Analysis of minimal complex systems and complex problem solving require different forms of causal cognition. Front. Psychol. 5:739. doi: 10.3389/fpsyg.2014.00739

Funke, J. (2014b). “Problem solving: what are the important questions?,” in Proceedings of the 36th Annual Conference of the Cognitive Science Society , eds P. Bello, M. Guarini, M. McShane, and B. Scassellati (Austin, TX: Cognitive Science Society), 493–498.

Funke, J., Fischer, A., and Holt, D. V. (2017). When less is less: solving multiple simple problems is not complex problem solving—A comment on Greiff et al. (2015). J. Intell. 5:5. doi: 10.3390/jintelligence5010005

Funke, J., Fischer, A., and Holt, D. V. (2018). “Competencies for complexity: problem solving in the 21st century,” in Assessment and Teaching of 21st Century Skills , eds E. Care, P. Griffin, and M. Wilson (Dordrecht: Springer), 3.

Funke, J., and Greiff, S. (2017). “Dynamic problem solving: multiple-item testing based on minimally complex systems,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 427–443.

Gobert, J. D., Kim, Y. J., Pedro, M. A. S., Kennedy, M., and Betts, C. G. (2015). Using educational data mining to assess students’ skills at designing and conducting experiments within a complex systems microworld. Think. Skills Creat. 18, 81–90. doi: 10.1016/j.tsc.2015.04.008

Goode, N., and Beckmann, J. F. (2010). You need to know: there is a causal relationship between structural knowledge and control performance in complex problem solving tasks. Intelligence 38, 345–352. doi: 10.1016/j.intell.2010.01.001

Gray, W. D. (2002). Simulated task environments: the role of high-fidelity simulations, scaled worlds, synthetic environments, and laboratory tasks in basic and applied cognitive research. Cogn. Sci. Q. 2, 205–227.

Greiff, S., and Fischer, A. (2013). Measuring complex problem solving: an educational application of psychological theories. J. Educ. Res. 5, 38–58.

Greiff, S., Fischer, A., Stadler, M., and Wüstenberg, S. (2015a). Assessing complex problem-solving skills with multiple complex systems. Think. Reason. 21, 356–382. doi: 10.1080/13546783.2014.989263

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2015b). Sometimes less is more: comparing the validity of complex problem solving measures. Intelligence 50, 100–113. doi: 10.1016/j.intell.2015.02.007

Greiff, S., Fischer, A., Wüstenberg, S., Sonnleitner, P., Brunner, M., and Martin, R. (2013a). A multitrait–multimethod study of assessment instruments for complex problem solving. Intelligence 41, 579–596. doi: 10.1016/j.intell.2013.07.012

Greiff, S., Holt, D. V., and Funke, J. (2013b). Perspectives on problem solving in educational assessment: analytical, interactive, and collaborative problem solving. J. Problem Solv. 5, 71–91. doi: 10.7771/1932-6246.1153

Greiff, S., Wüstenberg, S., Molnár, G., Fischer, A., Funke, J., and Csapó, B. (2013c). Complex problem solving in educational contexts—something beyond g: concept, assessment, measurement invariance, and construct validity. J. Educ. Psychol. 105, 364–379. doi: 10.1037/a0031856

Greiff, S., and Funke, J. (2009). “Measuring complex problem solving: the MicroDYN approach,” in The Transition to Computer-Based Assessment. New Approaches to Skills Assessment and Implications for Large-Scale Testing , eds F. Scheuermann and J. Björnsson (Luxembourg: Office for Official Publications of the European Communities), 157–163.

Greiff, S., and Funke, J. (2017). “Interactive problem solving: exploring the potential of minimal complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 93–105.

Greiff, S., and Martin, R. (2014). What you see is what you (don’t) get: a comment on Funke’s (2014) opinion paper. Front. Psychol. 5:1120. doi: 10.3389/fpsyg.2014.01120

Greiff, S., and Neubert, J. C. (2014). On the relation of complex problem solving, personality, fluid intelligence, and academic achievement. Learn. Ind. Diff. 36, 37–48. doi: 10.1016/j.lindif.2014.08.003

Greiff, S., Niepel, C., Scherer, R., and Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: an analysis of behavioral data from computer-generated log files. Comput. Hum. Behav. 61, 36–46. doi: 10.1016/j.chb.2016.02.095

Greiff, S., Stadler, M., Sonnleitner, P., Wolff, C., and Martin, R. (2017). Sometimes more is too much: a rejoinder to the commentaries on Greiff et al. (2015). J. Intell. 5:6. doi: 10.3390/jintelligence5010006

Greiff, S., and Wüstenberg, S. (2014). Assessment with microworlds using MicroDYN: measurement invariance and latent mean comparisons. Eur. J. Psychol. Assess. 1, 1–11. doi: 10.1027/1015-5759/a000194

Greiff, S., and Wüstenberg, S. (2015). Komplexer Problemlösetest COMPRO [Complex Problem-Solving Test COMPRO]. Mödling: Schuhfried.

Greiff, S., Wüstenberg, S., and Funke, J. (2012). Dynamic problem solving: a new assessment perspective. Appl. Psychol. Measure. 36, 189–213. doi: 10.1177/0146621612439620

Griffin, P., and Care, E. (2015). “The ATC21S method,” in Assessment and Taching of 21st Century Skills , eds P. Griffin and E. Care (Dordrecht, NL: Springer), 3–33.

Güss, C. D., and Dörner, D. (2011). Cultural differences in dynamic decision-making strategies in a non-linear, time-delayed task. Cogn. Syst. Res. 12, 365–376. doi: 10.1016/j.cogsys.2010.12.003

Güss, C. D., Tuason, M. T., and Orduña, L. V. (2015). Strategies, tactics, and errors in dynamic decision making in an Asian sample. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.13131

Güss, C. D., and Wiley, B. (2007). Metacognition of problem-solving strategies in Brazil, India, and the United States. J. Cogn. Cult. 7, 1–25. doi: 10.1163/156853707X171793

Herde, C. N., Wüstenberg, S., and Greiff, S. (2016). Assessment of complex problem solving: what we know and what we don’t know. Appl. Meas. Educ. 29, 265–277. doi: 10.1080/08957347.2016.1209208

Hermes, M., and Stelling, D. (2016). Context matters, but how much? Latent state – trait analysis of cognitive ability assessments. Int. J. Sel. Assess. 24, 285–295. doi: 10.1111/ijsa.12147

Hotaling, J. M., Fakhari, P., and Busemeyer, J. R. (2015). “Dynamic decision making,” in International Encyclopedia of the Social & Behavioral Sciences , 2nd Edn, eds N. J. Smelser and P. B. Batles (New York, NY: Elsevier), 709–714.

Hundertmark, J., Holt, D. V., Fischer, A., Said, N., and Fischer, H. (2015). System structure and cognitive ability as predictors of performance in dynamic system control tasks. J. Dynamic Deci. Mak. 1, 1–10. doi: 10.11588/jddm.2015.1.26416

Jäkel, F., and Schreiber, C. (2013). Introspection in problem solving. J. Problem Solv. 6, 20–33. doi: 10.7771/1932-6246.1131

Jansson, A. (1994). Pathologies in dynamic decision making: consequences or precursors of failure? Sprache Kogn. 13, 160–173.

Kaufman, J. C., and Beghetto, R. A. (2009). Beyond big and little: the four c model of creativity. Rev. Gen. Psychol. 13, 1–12. doi: 10.1037/a0013688

Knauff, M., and Wolf, A. G. (2010). Complex cognition: the science of human reasoning, problem-solving, and decision-making. Cogn. Process. 11, 99–102. doi: 10.1007/s10339-010-0362-z

Kretzschmar, A. (2017). Sometimes less is not enough: a commentary on Greiff et al. (2015). J. Intell. 5:4. doi: 10.3390/jintelligence5010004

Kretzschmar, A., Neubert, J. C., Wüstenberg, S., and Greiff, S. (2016). Construct validity of complex problem solving: a comprehensive view on different facets of intelligence and school grades. Intelligence 54, 55–69. doi: 10.1016/j.intell.2015.11.004

Kretzschmar, A., and Süß, H.-M. (2015). A study on the training of complex problem solving competence. J. Dynamic Deci. Mak. 1, 1–14. doi: 10.11588/jddm.2015.1.15455

Lee, H., and Cho, Y. (2007). Factors affecting problem finding depending on degree of structure of problem situation. J. Educ. Res. 101, 113–123. doi: 10.3200/JOER.101.2.113-125

Leutner, D., Fleischer, J., Wirth, J., Greiff, S., and Funke, J. (2012). Analytische und dynamische Problemlösekompetenz im Lichte internationaler Schulleistungsvergleichsstudien: Untersuchungen zur Dimensionalität. Psychol. Rundschau 63, 34–42. doi: 10.1026/0033-3042/a000108

Luchins, A. S. (1942). Mechanization in problem solving: the effect of einstellung. Psychol. Monogr. 54, 1–95. doi: 10.1037/h0093502

Mack, O., Khare, A., Krämer, A., and Burgartz, T. (eds) (2016). Managing in a VUCA world. Heidelberg: Springer.

Mainert, J., Kretzschmar, A., Neubert, J. C., and Greiff, S. (2015). Linking complex problem solving and general mental ability to career advancement: does a transversal skill reveal incremental predictive validity? Int. J. Lifelong Educ. 34, 393–411. doi: 10.1080/02601370.2015.1060024

Mainzer, K. (2009). Challenges of complexity in the 21st century. An interdisciplinary introduction. Eur. Rev. 17, 219–236. doi: 10.1017/S1062798709000714

Meadows, D. H., Meadows, D. L., and Randers, J. (1992). Beyond the Limits. Vermont, VA: Chelsea Green Publishing.

Meadows, D. H., Meadows, D. L., Randers, J., and Behrens, W. W. (1972). The Limits to Growth. New York, NY: Universe Books.

Meißner, A., Greiff, S., Frischkorn, G. T., and Steinmayr, R. (2016). Predicting complex problem solving and school grades with working memory and ability self-concept. Learn. Ind. Differ. 49, 323–331. doi: 10.1016/j.lindif.2016.04.006

Molnàr, G., Greiff, S., Wüstenberg, S., and Fischer, A. (2017). “Empirical study of computer-based assessment of domain-general complex problem-solving skills,” in The Nature of Problem Solving: Using research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 125–141.

National Research Council (2011). Assessing 21st Century Skills: Summary of a Workshop. Washington, DC: The National Academies Press.

Newell, A., Shaw, J. C., and Simon, H. A. (1959). A general problem-solving program for a computer. Comput. Automat. 8, 10–16.

Nisbett, R. E., and Wilson, T. D. (1977). Telling more than we can know: verbal reports on mental processes. Psychol. Rev. 84, 231–259. doi: 10.1037/0033-295X.84.3.231

OECD (2014). “PISA 2012 results,” in Creative Problem Solving: Students’ Skills in Tackling Real-Life problems , Vol. 5 (Paris: OECD Publishing).

Osman, M. (2010). Controlling uncertainty: a review of human behavior in complex dynamic environments. Psychol. Bull. 136, 65–86. doi: 10.1037/a0017815

Osman, M. (2012). The role of reward in dynamic decision making. Front. Neurosci. 6:35. doi: 10.3389/fnins.2012.00035

Qudrat-Ullah, H. (2015). Better Decision Making in Complex, Dynamic Tasks. Training with Human-Facilitated Interactive Learning Environments. Heidelberg: Springer.

Ramnarayan, S., Strohschneider, S., and Schaub, H. (1997). Trappings of expertise and the pursuit of failure. Simulat. Gam. 28, 28–43. doi: 10.1177/1046878197281004

Reuschenbach, B. (2008). Planen und Problemlösen im Komplexen Handlungsfeld Pflege. Berlin: Logos.

Rohe, M., Funke, J., Storch, M., and Weber, J. (2016). Can motto goals outperform learning and performance goals? Influence of goal setting on performance, intrinsic motivation, processing style, and affect in a complex problem solving task. J. Dynamic Deci. Mak. 2, 1–15. doi: 10.11588/jddm.2016.1.28510

Scherer, R., Greiff, S., and Hautamäki, J. (2015). Exploring the relation between time on task and ability in complex problem solving. Intelligence 48, 37–50. doi: 10.1016/j.intell.2014.10.003

Schoppek, W., and Fischer, A. (2015). Complex problem solving – single ability or complex phenomenon? Front. Psychol. 6:1669. doi: 10.3389/fpsyg.2015.01669

Schraw, G., Dunkle, M., and Bendixen, L. D. (1995). Cognitive processes in well-defined and ill-defined problem solving. Appl. Cogn. Psychol. 9, 523–538. doi: 10.1002/acp.2350090605

Schweizer, F., Wüstenberg, S., and Greiff, S. (2013). Validity of the MicroDYN approach: complex problem solving predicts school grades beyond working memory capacity. Learn. Ind. Differ. 24, 42–52. doi: 10.1016/j.lindif.2012.12.011

Schweizer, T. S., Schmalenberger, K. M., Eisenlohr-Moul, T. A., Mojzisch, A., Kaiser, S., and Funke, J. (2016). Cognitive and affective aspects of creative option generation in everyday life situations. Front. Psychol. 7:1132. doi: 10.3389/fpsyg.2016.01132

Selten, R., Pittnauer, S., and Hohnisch, M. (2012). Dealing with dynamic decision problems when knowledge of the environment is limited: an approach based on goal systems. J. Behav. Deci. Mak. 25, 443–457. doi: 10.1002/bdm.738

Simon, H. A. (1957). Administrative Behavior: A Study of Decision-Making Processes in Administrative Organizations , 2nd Edn. New York, NY: Macmillan.

Sonnleitner, P., Brunner, M., Keller, U., and Martin, R. (2014). Differential relations between facets of complex problem solving and students’ immigration background. J. Educ. Psychol. 106, 681–695. doi: 10.1037/a0035506

Spering, M., Wagener, D., and Funke, J. (2005). The role of emotions in complex problem solving. Cogn. Emot. 19, 1252–1261. doi: 10.1080/02699930500304886

Stadler, M., Becker, N., Gödker, M., Leutner, D., and Greiff, S. (2015). Complex problem solving and intelligence: a meta-analysis. Intelligence 53, 92–101. doi: 10.1016/j.intell.2015.09.005

Stadler, M., Niepel, C., and Greiff, S. (2016). Easily too difficult: estimating item difficulty in computer simulated microworlds. Comput. Hum. Behav. 65, 100–106. doi: 10.1016/j.chb.2016.08.025

Sternberg, R. J. (1995). “Expertise in complex problem solving: a comparison of alternative conceptions,” in Complex Problem Solving: The European Perspective , eds P. A. Frensch and J. Funke (Hillsdale, NJ: Erlbaum), 295–321.

Sternberg, R. J., and Frensch, P. A. (1991). Complex Problem Solving: Principles and Mechanisms. (eds) R. J. Sternberg and P. A. Frensch. Hillsdale, NJ: Erlbaum.

Strohschneider, S., and Güss, C. D. (1998). Planning and problem solving: differences between brazilian and german students. J. Cross-Cult. Psychol. 29, 695–716. doi: 10.1177/0022022198296002

Strohschneider, S., and Güss, C. D. (1999). The fate of the Moros: a cross-cultural exploration of strategies in complex and dynamic decision making. Int. J. Psychol. 34, 235–252. doi: 10.1080/002075999399873

Thimbleby, H. (2007). Press On. Principles of Interaction. Cambridge, MA: MIT Press.

Tobinski, D. A., and Fritz, A. (2017). “EcoSphere: a new paradigm for problem solving in complex systems,” in The Nature of Problem Solving: Using Research to Inspire 21st Century Learning , eds B. Csapó and J. Funke (Paris: OECD Publishing), 211–222.

Tremblay, S., Gagnon, J.-F., Lafond, D., Hodgetts, H. M., Doiron, M., and Jeuniaux, P. P. J. M. H. (2017). A cognitive prosthesis for complex decision-making. Appl. Ergon. 58, 349–360. doi: 10.1016/j.apergo.2016.07.009

Tschirgi, J. E. (1980). Sensible reasoning: a hypothesis about hypotheses. Child Dev. 51, 1–10. doi: 10.2307/1129583

Tuchman, B. W. (1984). The March of Folly. From Troy to Vietnam. New York, NY: Ballantine Books.

Verweij, M., and Thompson, M. (eds) (2006). Clumsy Solutions for A Complex World. Governance, Politics and Plural Perceptions. New York, NY: Palgrave Macmillan. doi: 10.1057/9780230624887

Viehrig, K., Siegmund, A., Funke, J., Wüstenberg, S., and Greiff, S. (2017). “The heidelberg inventory of geographic system competency model,” in Competence Assessment in Education. Research, Models and Instruments , eds D. Leutner, J. Fleischer, J. Grünkorn, and E. Klieme (Heidelberg: Springer), 31–53.

von Clausewitz, C. (1832). Vom Kriege [On war]. Berlin: Dämmler.

Wendt, A. N. (2017). The empirical potential of live streaming beyond cognitive psychology. J. Dynamic Deci. Mak. 3, 1–9. doi: 10.11588/jddm.2017.1.33724

Wiliam, D., and Black, P. (1996). Meanings and consequences: a basis for distinguishing formative and summative functions of assessment? Br. Educ. Res. J. 22, 537–548. doi: 10.1080/0141192960220502

World Economic Forum (2015). New Vsion for Education Unlocking the Potential of Technology. Geneva: World Economic Forum.

World Economic Forum (2016). Global Risks 2016: Insight Report , 11th Edn. Geneva: World Economic Forum.

Wüstenberg, S., Greiff, S., and Funke, J. (2012). Complex problem solving — more than reasoning? Intelligence 40, 1–14. doi: 10.1016/j.intell.2011.11.003

Wüstenberg, S., Greiff, S., Vainikainen, M.-P., and Murphy, K. (2016). Individual differences in students’ complex problem solving skills: how they evolve and what they imply. J. Educ. Psychol. 108, 1028–1044. doi: 10.1037/edu0000101

Wüstenberg, S., Stadler, M., Hautamäki, J., and Greiff, S. (2014). The role of strategy knowledge for the application of strategies in complex problem solving tasks. Technol. Knowl. Learn. 19, 127–146. doi: 10.1007/s10758-014-9222-8

Keywords : complex problem solving, validity, assessment, definition, MicroDYN

Citation: Dörner D and Funke J (2017) Complex Problem Solving: What It Is and What It Is Not. Front. Psychol. 8:1153. doi: 10.3389/fpsyg.2017.01153

Received: 14 March 2017; Accepted: 23 June 2017; Published: 11 July 2017.

Reviewed by:

Copyright © 2017 Dörner and Funke. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Joachim Funke, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Complex or 'Wicked Issues'

Complex problems are typically defined as those that include the ability to approach them from multiple, sometimes competing, perspectives and which may have multiple possible solutions.

• Agreement on the definition of the problem (owing to multiple values, perception, and perspectives)
• Clear solutions to the problem owing to the wide array of possible solutions and trade-offs associated with each
• Easily identified causes or authority due to the problem having multiple potential causes, jurisdictions, stakeholders and regulator or implications

Additionally, the solution to such problems may result in unintended consequences that require re-solution of the new problem. Since 1968, this type of problem has also been referred to as 'wicked problems' (REF). The term 'wicked problem' is primarily attributed to an article by Rittel and Webber in 1973 about problems in planning. While some would say wickedness simply means complex, Conklin suggests that Rittel and Webber: ' distinguished a new domain of problem type, as opposed to, say, a new way of solving complex problems. Problem wickedness is not about a higher degree of complexity, it is about a fundamentally different kind of challenge to the design process, one that makes solution secondary and problem understanding central.' (Conklin et al, 2007).

Conklin (2007) goes on to state that Rittel and Webber's (1973) conceptualization of problems is one that cannot be arrived at using the traditional linear modes of problem solving and, in particular, arrived at in a way of re-incorporating the human dimension of problems back into problem solving approaches. Previously, traditions of rationalist problem solving had sought to remove the social or human from problems and create a vacuum within which problem-solving occurred. A similar rationalization is found in traditional scientific, positivist approaches to research, development and extension ii .

Compounding these challenges is that fact that in almost all cases the conditions and constraints affecting the problem and its possible solutions change over time, sometimes often dramatically - changing both the problem and the range of options designed to address it.

i Effective Community Engagement: Workbook and Tools, Version 2, Department of Sustainability and Environment , Department of Primary Industries.

ii Community Engagement Scoping Paper, Thompson, Lyndal 2014

• Degrees & Programs
• College Directory

Information for

• Faculty & Staff
• Visitors & Public