Integration of prior knowledge during haptic exploration depends on information type

Abstract

When haptically exploring softness, humans use higher peak forces when indenting harder versus softer objects. Here, we investigated the influence of different channels and types of prior knowledge on initial peak forces. Participants explored two stimuli (hard vs. soft) and judged which was softer. In Experiment 1 participants received either semantic (the words “hard” and “soft”), visual (video of indentation), or prior information from recurring presentation (blocks of harder or softer pairs only). In a control condition no prior information was given (randomized presentation). In the recurring condition participants used higher initial forces when exploring harder stimuli. No effects were found in control and semantic conditions. With visual prior information, participants used less force for harder objects. We speculate that these findings reflect differences between implicit knowledge induced by recurring presentation and explicit knowledge induced by visual and semantic information. To test this hypothesis, we investigated whether explicit prior information interferes with implicit information in Experiment 2. Two groups of participants discriminated softness of harder or softer stimuli in two conditions (blocked and randomized). The interference group received additional explicit information during the blocked condition; the implicit-only group did not. Implicit prior information was only used for force adaptation when no additional explicit information was given, whereas explicit interfered with movement adaptation. The integration of prior knowledge only seems possible when implicit prior knowledge is induced—not with explicit knowledge.

Introduction

In everyday life, we manually interact with many different objects, and haptic information plays a key role in this process. We use haptic information to plan and guide a variety of actions, or just tactually explore object properties. However, interacting with objects does not start at first touch. Even before the first haptic contact, we typically look at objects. Sometimes we have been told about object properties, or we have previous experience with objects of the same or similar type (Ernst & Bülthoff, 2004; Newell, Ernst, Tjan, & Bülthoff, 2001). All this prior knowledge potentially facilitates interactions with objects. For example, when lifting objects, humans have to estimate the object's weight to adjust their grip and lifting forces (Johansson & Cole, 1992). For this estimation they have been shown to rely on prior knowledge before the interaction and to integrate sensory feedback later (Flanagan & Wing, 1997; Wolpert & Flanagan, 2001). For the haptic exploration process, it has been shown that prior knowledge from previous experiences with an object category is used in order to improve exploratory behavior and perception (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). Given that exploring objects and interacting with them is such an essential part of human behavior, we would expect that any available prior information is used to improve object manipulation. In this study we tested this prediction by investigating how different types of prior information influence haptic exploration behavior in softness discrimination. 

The physical correlate of softness is compliance, which is a property that is highly relevant in everyday behavior and is better accessed haptically than by any other sense (Klatzky & Lederman, 1999). Compliance is defined as the extent of object deformation under a given force (mm/N). People manually explore object compliance typically by applying a normal force to the surface while it is in a fixed position (Lederman & Klatzky, 1987). In this process, humans integrate cutaneous information provided by the mechanoreceptors in the skin, like surface and finger pad deformation. They also utilize vibrotactile information from initial contact, and kinesthetic information provided by mechanoreceptors in the muscles, like muscle tension, or stretch in ligaments (Klatzky & Lederman, 1999). The contribution of different cues changes with the object's properties: For more compliant objects (as used in the present experiments) finger deformation and contact area are major cutaneous cues in the perception of softness, whereas for very hard objects, vibrotactile information becomes dominant (Friedmann, Hester, Green, & LaMotte, 2008; Srinivasan & LaMotte, 1995; Visell & Okamoto, 2014). 

One way to apply normal forces in compliance exploration is to squeeze the object between the thumb and another finger (Tan, Durlach, Beauregard, & Srinivasan, 1995). Alternatively, in particular when an object is too heavy or too large to lift, or when it is in a fixed position by itself, humans indent the object's surface against a support (e.g., a table). Typically, participants increase and decrease the normal forces repetitively several times (Kaim & Drewing, 2011; Lederman & Klatzky, 1987). By doing this, humans collect sensory information which is integrated over the exploration process (Klatzky & Lederman, 1999; Lederman & Klatzky, 1987; Lederman & Klatzky, 2009; Lezkan & Drewing, 2018; Metzger, Lezkan, & Drewing, 2018; Srinivasan & LaMotte, 1995). 

It has been suggested that sensory information and motor commands influence each other in a closed-loop feedback scheme during the exploration process (Saig, Gordon, Assa, Arieli, & Ahissar, 2012). This means that sensory information is used to improve the efficiency of exploratory movements, which in turn improve the gathering of further information (Lezkan & Drewing, 2015; Lezkan, Metzger, & Drewing, 2018). By this interaction humans adapt to the conditions of the task and obtain information in the most effective way. According to the model suggested by Lezkan and Drewing (2015; Figure 1), adaptation of motor commands to e.g., the compliance of explored objects is based on both gathered sensory information (tactile and kinesthetic information) and prior knowledge. Kaim and Drewing (2011) showed that exploration behavior in softness discrimination is adapted based on prior knowledge and that this adaptation enhances perceptual performance. In one experiment, participants compared the softness of two stimuli in a 2AFC discrimination task. Stimuli in a pair were either both relatively hard or both relatively soft. Participants received no prior information on the compliance category of the upcoming pair (order of pairs randomized) or prior information from recurring compliance (stimuli were presented in blocks of either softer or harder pairs only). The influence of prior information was measured through the amount of maximal applied force at the initial indentation of the first stimulus (initial peak force). The initial peak force can be considered to be mainly driven by prior knowledge, because initial peak force is programmed at a time point when sensory information on compliance hardly seems to be available (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). When an object's compliance category was predictable, participants adapted their initial peak force: They systematically used higher initial forces for harder versus softer stimuli, whereas no difference between the initial forces for harder and softer stimuli was observed in randomized presentation. Lezkan and Drewing (2015) and Lezkan, Metzger, and Drewing (2018) showed that in later indentations, both prior information and already gathered sensory information are integrated to control the exploration behavior. However, in that study, prior knowledge remained highly relevant for motor control over the whole exploration process, although more and more sensory information was available. While these results suggest that prior information plays a core role in the control of exploratory behavior, they are limited to a single specific type of prior information, namely recurring stimuli presentation. 

Figure 1

 

Sensorimotor control loop for active haptic exploration adapted from (Lezkan & Drewing, 2015) and potential influence of prior knowledge indicated in this study.

Ernst and Bülthoff (2004) emphasize that natural perception relies on a multitude of information sources, including different sources of prior information. In natural settings, prior information on an upcoming object is not only given through repeated presentation of similar objects over a certain period of time. Humans often have expectations from earlier interactions with an object, or an object of a similar type, which can be evoked when we are told about the identity or properties of an object. In many situations we also first look at an object before we interact with it haptically (Ernst & Bülthoff, 2004; Newell et al., 2001). For humans, written and spoken communication is the primary basis for sharing information (Cherry, 1957) so that verbal information about an object is often available before interacting with it. Visual information on the other hand is very tightly connected with haptic information and in many tasks an interaction of the two is required. For example, the connection is highlighted by the tight interaction of haptic and visual information during the exploration of objects. While interacting with objects, humans mostly see the front side, but haptically sense the backside (Newell et al., 2001). This leads to a complementary viewpoint-dependent representation of the explored object in both senses. Newell et al. (2001) argued that the two senses then tightly interact to form an optimal representation of the explored object. 

In addition, as with other information, prior information can be implicit or explicit. Depending on the information type, humans can learn implicitly and explicitly. When explicit information is present, humans can learn consciously by knowing facts about the properties of an object and acquire explicit knowledge. Given implicit information, humans can learn implicitly, without knowing facts or rules. In this case systematic patterns as for example the sequence of given objects is learned without the subject being aware of them (Dienes & Perner, 1999; Sun, Slusarz, & Terry, 2005). When having enough time, implicit learning can lead to explicit knowledge and explicit learning can lead to implicit knowledge as it does, for instance, when learning motion sequences in sport (Anderson, 1983; Masters, 1992). This definition of explicit and implicit knowledge is in line with existing theories, such as with the distinction between declarative and procedural knowledge as defined by many (e.g., Anderson, 1983, 1993) and the theory of controlled and automatic processing in tasks (Fisk, Ackerman, & Schneider, 1987). 

In the present study, we asked how prior information given through different channels (recurring compliance, semantic and visual channels) and of different type (implicit, explicit) is integrated in the exploration process and whether it would similarly serve motor adaptation as was observed for recurring compliance. In the first experiment, we varied the channel of prior information (note besides that data of the first experiment has been prepublished in a conference article, Zoeller, Lezkan, Paulun, Fleming, & Drewing, 2018). Because in our everyday life visual and semantic information is highly relevant (Ernst & Bülthoff, 2004; Newell et al., 2001), we choose to compare their influence on haptic exploration to the previously observed influence of prior information from recurring compliance. In the experiment, participants discriminated either two relatively hard or two relatively soft stimuli according to softness. Prior knowledge on compliance category was induced semantically by displaying the words “soft” or “hard” on the screen; visually by a short movie of the indentation of a softer or harder stimulus preceding the exploration; by recurring blocked presentation of stimuli from the same compliance category; or no prior information was given (random order of harder and softer pairs). In the visual condition, stimuli were rendered physical simulations of a nonrigid object indented by a rod. Paulun, Schmidt, van Assen, and Fleming (2017) found that visual softness ratings for these stimuli can be predicted by a simple heuristic: The more an object was deformed, the softer it was perceived. We exploited this to visually indicate object compliance. We then systematically compared the differences of initial peak forces for harder versus softer stimuli. We expected higher initial peak forces for harder stimuli as compared to softer ones, when prior information is given, independent of the channel. 

However, we found the expected result only in the conditions with repeated stimulus presentation, but not in the semantic or visual prior knowledge conditions. We wondered whether semantic and visual prior information may have failed to induce motor adaptation because both information channels provided information in an explicit manner: Semantic and visual prior information told or showed facts about the properties of an object, which can be learned consciously and should hence lead to explicit knowledge. In contrast, when information is given through stimulus repetition, this method is not known to the participants nor are they immediately aware. Hence it is rather learned implicitly, resulting in implicit knowledge. The two states of knowledge have been shown to have different influences on behavior in motor tasks. Additional explicit knowledge during a task conducted implicitly, as in highly trained motor skills, can interfere with the task and deteriorate performance (Masters, 1992), and implicit and explicit information can be processed differently, which could also lead to different influences on exploration behavior (Easton, Greene, & Srinivas, 1997). In Experiment 1, however, multiple variables differed between the presumably implicit and the explicit conditions, including, for example, presentation order (blocked vs. randomized in semantic and visual condition), or the way to present compliance information (repetition vs. naming and presenting an action). Thus, to examine the influence of explicit information more directly, we conducted Experiment 2, in which we only manipulated the type of knowledge during the same task. Specifically, in Experiment 2 we tested whether explicit knowledge may interfere with the positive influence of implicit knowledge on exploratory adaptation. Half of the participants received only implicit prior knowledge about the compliance of upcoming stimuli (blocked presentation of softer or harder stimuli); the other half of the participants additionally received written explicit information before each block of harder or softer stimuli via the sentences “The upcoming stimuli will be soft!” and “The upcoming stimuli will be hard!” To both groups we also presented a control condition without any prior information on compliance. 

Experiment 1

In Experiment 1 we tested how the channel of prior knowledge influences motor behavior in softness discrimination. We hypothesized that humans use available prior information of any source to adapt their forces to the compliance of the exposed stimuli. Participants received prior information in four conditions (semantic, visual, stimulus repetition, control). Previously it was observed that humans use higher peak forces in their first indentation for harder versus softer objects when presenting stimuli in blocks of the same compliance category and that this motor adaptation improves perception (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). We expected this motor adaption when any prior information regarding the compliance category is given, even if the effect size might differ with the information channel and representation (cf. Easton et al., 1997). 

Method

Participants

Twenty-four healthy students from the Justus-Liebig University in Giessen participated (three males, 21 females; average age: 23.8, range: 19–30). Only participants who reported no tenosynovitis in the past, and showed no motor or cutaneous impairments were included. All participants had normal or corrected-to-normal vision. One participant was left-handed; all others were right-handed. Participants were naïve to the purpose of the experiment, provided written informed consent, and were paid for participating. Methods and procedures of the study were approved by the local ethics committee LEK FB06 at Giessen University, and conducted in accordance with the ethical standards laid down in the 2008 Declaration of Helsinki. 

Setup and stimuli

During the experiment participants sat at a custom-made visuo-haptic workbench (Figure 2), consisting of a PHANToM 1.5A haptic force feedback device (spatial resolution: 0.03 mm, temporal resolution: 1000 Hz, used to collect positional data), a force sensor to collect data of the executed finger force (682 Hz, resolution: 0.05 N), and a 22-in. 3D computer screen (120 Hz, 786 × 1024 pixels). Participants looked at the screen through a mirror (viewing distance 40 cm), which displayed a visual scene aligned with the haptic workspace. The mirror prevented participants from seeing their hand. In the visual scene, stimuli were displayed as blue cubes on a green and black checkerboard (Paulun et al., 2017). The position of the cubes in the visual scene was identical to the stimulus position on the table. Two stimuli were always placed side-by-side on the force sensor in front of the participant. During the task, a small sphere (8 mm diameter) represented the finger position of the participant in the scene. When participants indented the stimulus surfaces, the sphere disappeared to give no visual feedback. Visual stimuli were presented in 3D using stereo glasses (Nvidia 3D Vision 2). We used a chinrest to stabilize participants' head position. The index finger of participants was connected to the force feedback device via a spherical magnetic fixed at the fingernail. This allowed participants to move their finger in all axes with the maximum amount of freedom in a 38 × 27 × 20 cm³ haptic workspace. The adapter also left the fingertip free to allow for bare-finger exploration. Devices were connected to a PC where custom-made software collected the data and controlled the experiment. Noises produced by the PHANToM and the computers cooling system were masked with noise-canceling headphones. 

Figure 2

 

Schema of the custom made visuo-haptic workbench.

As haptic stimuli we used six different custom-made rubber disc silicon stimuli with a height of 38 mm and a diameter of 75 mm, which varied in compliance (see Kaim & Drewing, 2011 for details of the production process). Stimuli were divided in a hard category (0.41 mm/N, 0.45 mm/N, 0.49 mm/N), and a soft category (0.91 mm/N, 0.95 mm/N, 1.04 mm/N). In each category the stimulus with the middle compliance was used as standard stimulus and the other two (one harder one softer) as comparison stimuli. In each trial participants compared the standard stimulus to one of the comparison stimuli within the same category. 

In order to determine haptic standard stimuli and good visual predictors of the haptic stimuli, in a first pilot study (N = 12) we had carefully matched visually displayed compliance to haptically felt compliance. We presented fifteen videos of a probe indenting a deformable object. Videos differed in the maximum amount of deformation. The deformation displayed in the videos ranged from rather hard (1.6 mm deformation) to soft (33.6 mm deformation) in 15 steps. Ten silicone rubber disc stimuli (0.15 mm/N to 1.04 mm/N in steps of ∼0.1 mm/N) were presented in a randomized order on a wheel (Figure 3) and inside a box. The order stayed the same for all trials of a single participant. Participants first watched one of the videos. Then they had to match the visually perceived softness to the compliance of a rubber stimulus, by choosing the one that fit best. Participants were well able to discriminate between compliances and to match visual and physical compliances (Figure 4), which was evident from a good fit of a power function between the average rated compliance of the physical rubber stimuli and the displayed visual deformation (R² = 0. 99). As haptic standard stimuli for the main experiment, we chose two stimuli from the pilot set (0.45 mm/N and 0.95 mm/N) that were most often associated with low (4.8 mm) and high (33.6 mm) visual deformation. As visual predictors we chose the matched video sequences. Comparison stimuli were determined in a second pilot study, in which eight participants compared the two standard stimuli each with a set of four preselected stimuli in a 2AFC discrimination task (96 trials, random order of stimuli from different compliance categories). Comparison stimuli with an average chance of approximately 80% (softer: 82%, 77%; harder: 83%, 79%) for giving the correct answer were picked. 

Figure 3

 

Schema of the equipment of the Pilot study one. Participants reached through the hole in the front of the box to explore stimuli inside. Only one stimulus could be touched at a time.

Figure 4

 

Average matched physical compliance (mm/N) and standard error of the mean (SEM) as a function of the shown visual deformation (mm). The line represents the fitted power function.

Design and procedure

The experiment included the within-participant variables Compliance Category (harder vs. softer) and Prior Knowledge (recurring compliance, semantic, visual, control). In three Prior Knowledge conditions, prior knowledge on the compliance category of the upcoming stimulus pair was induced: The recurring compliance condition was divided in blocks of trials where either only harder or softer pairs of stimuli were repeatedly presented in order to induce knowledge about the upcoming pair of stimuli (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). In the three other conditions, trials with pairs of softer and harder stimuli were presented in random order. In the semantic condition, prior knowledge was induced with the German words Weich or Hart (soft, hard) predicting the compliance of the upcoming pair of stimuli. In the visual condition, a video of a probe indenting into the surface of a deformable object was shown in the beginning of every trial. Displaying prior knowledge for the softer category, the probe deformed the objects surface quite far, whereas for the harder category, it stopped early (Figure 5). The position of the visual object was identical to the position of the starting stimulus in the following exploration task. In the control condition no prior knowledge was acquired. 

Figure 5

 

Example of the maximal amount of deformation displayed in the visual condition (right picture: softer condition; left picture: harder condition).

Participants conducted a two alternative forced-choice discrimination task. On each trial, participants explored the standard stimulus and one of the two comparison stimuli of one compliance category. Standard and comparison stimuli were equally often assigned as starting stimulus. The position of the starting stimulus (left or right) was equally often right and left in randomized order. After exploring, participants had to judge, which of the stimuli felt softer. 

During the semantic and the visual condition, prior information was displayed for 3.5 s at the beginning of each trial. In each condition a screen followed, showing a representation of one of the two stimuli (blue cube), indicating to the participants where they should start the haptic exploration. The exploration phase was initiated by a beep sound. As soon as participants touched the surface of the starting stimulus, a representation of the second stimulus appeared on the screen. Participants were instructed to use the typical movement scheme for their exploration, i.e., pressing in normal direction into the surface (Lederman & Klatzky, 1987). At the same time, they were free to switch between stimuli and to indent each stimulus as often as desired. Participants responded by pressing a virtual button above the stimulus they perceived as softer. Because stimuli had to be changed manually after every trial, participants had to move their finger to a specified waiting position. Stimuli were chosen to be about 80% distinguishable. Due to this modification, participants were likely to be constantly uncertain about their decisions. To maintain participants' motivation, feedback about the performance was given at the end of every condition, by displaying the percentage of correct responses on the screen for 3 s. Because feedback was only given after completing a condition, it was neither informative about specific judgments, nor could it be used be used to improve performance in that condition. 

The experiment took on average 2.5 hrs in total. It was conducted in two sessions (each 1.25 hr), each of which contained two of the four conditions. Across participants we balanced the order of conditions by using a Latin square (4 × 4), to exclude session and other order effects. Further, the order of compliance categories in the implicit condition was balanced. Half of the participants in each sequence of the Latin square started with the softer category, and the other half with the harder category. Each condition consisted of 48 trials (12 per comparison), so that each participant conducted 192 trials. We implemented a break of 60 s in the middle of each condition, and a break of at least 120 s between conditions. Before starting with the main experiment, participants performed eight test trials with no prior knowledge presenting stimuli that were not used in the main experiment, to familiarize themselves with the task. 

Data analysis

We focused on analyzing the initial peak forces (Figure 6) in the exploration of each stimulus pair. Initial peak force seems to play an important role in softness perception (Srinivasan & LaMotte, 1995), and is a good indicator of the influence of prior knowledge on motor behavior because it can be assumed to be hardly affected by sensory feedback (Kaim & Drewing, 2011). To capture the initial peak force of each trial, we subtracted stimulus mass from force measurements and smoothed the resulting force values with a moving-averaging window with a kernel of 45 ms. We then identified the first turning point in which the derivative of force over time changed from positive to negative in the trajectories and the maximum applied force was higher than 3N. The time interval between two turning points was restricted to be at least 180 ms to exclude local maxima, small finger shaking movements, or movement rests while releasing the finger from the object after valid peaks. If more than one peak appeared within the timeframe, only the highest one was assigned to be the maximum force. 

Figure 6

 

Example of the raw force data (N) gathered over time and the smoothed force (N) from a complete trial. Depicted are four indentations of the two stimuli. For the analysis only initial peak forces were used.

Initial peak forces were compared via a repeated measurement ANOVA with the variables Compliance Category (softer, harder) and Prior Knowledge (visual, recurring, semantic, and control). We expected that participants apply more force in the first indentation when exploring harder as compared to softer stimuli in the three test conditions, leading to higher hard-soft differences in the test conditions as compared to the randomized control condition. We compared these hard-soft differences between the three test conditions and the control condition via one-sided planned t tests. All comparisons were planned based on straightforward theoretical predictions and thus do not need correction for multiple testing (Keppel & Wickens, 2004; Saville, 1990). In particular, we focused on the hard-soft difference, because this differentiation of exploration behavior unequivocally indicates the use of predictive signals concerning compliance categories. We further analyzed data that was not in line with our directed hypothesis posthoc with two-sided t tests. 

Results

In the ANOVA on initial peak forces (Figure 7), we found no significant main effect for Compliance Category, F(1, 23) = 0.009 p = 0.924, no significant main effect for Prior Knowledge, F(1, 23) = 0.389, p = 0.761, but the expected significant interaction of the two factors, F(1, 23) = 5.270, p = 0.002. By calculating planned t tests (Keppel & Wickens, 2004; Saville, 1990), we compared the differences between forces applied to harder minus softer stimuli in all three test conditions to the corresponding hard-soft difference in the randomized control condition. In line with our hypothesis, we found a significant effect between prior information from recurring compliance and the randomized control condition, t(23) = 1.924, p = 0.034 (one-sided). In contrast to our hypothesis, we found no effect between the semantic and random conditions, t(23) = 0.229, p = 0.411 (one-sided). We also did not find the expected effect between the visual and the random condition, but a posthoc t test showed an unexpected effect between the visual and the random condition, t(23) = −2.193, p = 0.039 that indicates a significantly larger difference in the visual condition, but with more force used for softer stimuli. 

Figure 7

 

Average initial peak force (N) and ipsative standard error of the mean (SEM; Loftus & Masson, 1994) used in the first indentation of stimuli of the harder and softer category, in each prior knowledge condition.

On average, participants performed 14.0 indentations and changed 4.5 times between stimuli per trial. Participants answered on average on 86% of the trials correct. Individual performance ranged from 75% to 94%. We found no difference between softer (86%) and harder stimuli (85%), t(23) = 1.167, p = 0.255 (based on angular transformed values; Claringbold, Biggers, & Emmens, 1953)). In the visual condition participants responded correct 84% of the time, in the semantic condition 88%, and in the recurring condition 87%. Performance in the control condition was 84%. 

Discussion: Experiment 1

In Experiment 1 we compared the influence of different sources of prior knowledge on exploratory forces in softness discrimination. Our hypothesis was that independent of the source of knowledge, participants would use larger peak forces at first touch, when prior knowledge indicates that the following stimuli will be harder rather than softer. Previous studies had demonstrated that such motor adaptation occurs with prior knowledge from recurring stimulus compliance and serves to improve perception (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). In the analysis we focused on peak-force differences between the compliance categories. Our results confirm that when receiving prior knowledge from stimulus repetition, the hard-soft difference was larger than in the control condition. When comparing the semantic and the control condition, we found no effects for peak force differences. Surprisingly, when receiving visual prior information, participants used less force at first touch when exploring harder stimuli. The hard-soft difference of the visual condition was significantly larger than in the control condition. We conclude that the prior knowledge gathered through visual and semantic channels in the present experiment did not allow for an adaptation of exploratory behavior. 

Why might this be the case? We speculate that a successful adaptation of exploration behavior may depend on whether the induced prior knowledge is of implicit or explicit type. During blocked presentation, prior knowledge is built up implicitly through recurring presentation of stimuli with similar compliance. In contrast, in the semantic and the visual condition facts about the compliance category were displayed by lexical definitions and clearly distinguishable visual deformations, respectively, so that the prior information was provided explicitly, resulting in explicit prior knowledge. It was reported that highly trained motor skills, like basic haptic exploration behavior, (Sun, Merrill, & Peterson, 2001), are conducted using mainly implicit knowledge (Lederman & Klatzky, 1987), whereas explicit knowledge can interfere with automatic responses (Masters, 1992). In addition, behavioral and neuroimaging studies suggest differences between the processing of implicit knowledge and of the corresponding explicit knowledge (Easton et al., 1997; Pascual-Leone, Grafman, & Hallett, 1994). In the present experiment, inducing explicit knowledge in the semantic and visual conditions might have induced a consciously controlled process, which hindered a proper integration of prior information as in a natural unconscious automatic exploration process. That is, the mainly implicit exploratory behavior may be not affected by—or even disturbed by—essentially useful but explicit prior knowledge. Differences in the semantic and the visual condition might be due to differences in the use and quality of the explicit information. One explanation for the lack of effect in the semantic condition may be a lack of correspondence of the representation of semantic and haptic information. Former studies suggest that haptic and visual information are closely related and similarly represented, whereas verbal information can be processed and represented in a quite different way (Easton et al., 1997; Goodale & Miller, 1992; Newell et al., 2001). Possibly reduced cross-modal conformity could lead to less integration of the prior information in the exploration process. The closer connection between the haptic and visual sense (Newell et al., 2001) and the more precise presentation of stimulus compliance in the visual as compared to the semantic condition may have led to a more intensive use of explicit information in the visual as compared to the semantic condition and thus to a higher interference and a stronger (maladaptive) effect on the exploration behavior. 

However, from the results of Experiment 1 alone we cannot unequivocally draw conclusions on a difference between the processing of implicit and explicit prior knowledge in haptic exploration. Multiple variables differed between the conditions, such as presentation order (blocked vs. randomized), or the way to present compliance information (repetition vs. naming vs. presenting an action). We therefore examined the influence of explicit prior information more specifically in the Experiment 2. 

Experiment 2

In Experiment 2, we focused on the influence of implicit and explicit prior information on exploration behavior and if this influence could explain the unexpected exploration behavior in Experiment 1. We varied only the available types of prior information. Again, participants compared the softness of two stimuli, and we focused at the peak forces of the first indentations. All participants performed a control condition without prior knowledge, where trials with softer and harder stimuli were presented in random order, and a test condition, where prior information was given implicitly by blocked presentation of harder or softer stimuli. Participants were split in two groups: Participants in the implicit-only group executed the test and control condition similar to the implicit and control condition in Experiment 1 and previous studies (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). In the interference group additional explicit information was given before each block of the test condition by displaying the sentences “The upcoming stimuli will be soft!” or “The upcoming stimuli will be hard!” Blocks of test and control condition contained only 16 trials and were presented in random order, to induce relearning in each block and prevent long-term effects of gathered knowledge (e.g., state changes from implicit to explicit in the implicit-only group). If explicit knowledge interferes with the exploration process, we should find adaptation of initial peak forces in the implicit-only group (Lezkan & Drewing, 2015) when inducing implicit prior knowledge, but no adaptation when adding explicit knowledge. 

Methods

Participants

Twenty-four healthy students from the Justus-Liebig-University in Giessen participated (seven males, 17 females; average age in years: 23.9, range: 19–29). Participants were randomly assigned to two groups (implicit-only group: average age 23.6 years, nine females; interference group: 24.2 years, eight females) All participants were right-handed and had normal or corrected-to-normal vision. Participants were naïve to the purpose of the experiment and were paid for participating. Exclusion conditions were equal to Experiment 1. 

Setup and stimuli

Participants performed the experiment using the same setup and rubber stimuli of the same type as in Experiment 1. The harder set in Experiment 2 included three stimuli with compliances 0.14 mm/N (standard), 0.12 mm/N, and 0.16 mm/N; the softer set included 0.95 mm/N (standard), 0.91 mm/N, and 1.04 mm/N. Pilot data show that participants correctly discriminated between standard and comparison stimuli with about 80% probability. 

Design and procedure

The experiment included a test condition, where prior information was available, and a control condition. Participants were split into two groups, each of which performed a test and a control condition. In the control condition stimuli were presented in a random order, so that no prior knowledge was induced. In the test condition stimuli from the harder or softer condition only were presented in blocked presentation to induce implicit prior knowledge. In the interference group, additional explicit knowledge was induced before each block: For that purpose, the sentences “Die folgenden Reize werden hart [sein]” (the following stimuli will be hard) or “Die folgenden Reize werden weich [sein]” (the following stimuli will be soft) were displayed for 15 s immediately before each block. For the implicit-only group no additional information was presented before any block. Before the experiment, all participants were informed that the experiment contained twelve blocks of sixteen trials each. In the interference group, participants were additionally informed that before some, but not all blocks, important written information for the following stimuli would be given. Participants were explicitly instructed to pay attention to this information. Task and exploration conditions were equal to those of Experiment 1. 

The experiment consisted of 12 blocks (three hard only blocks, three soft only blocks, and six randomized blocks) each containing 16 trials. We presented blocks from test and control conditions in randomized order. We conclude from a posthoc analysis that participants were able to gather sufficient implicit knowledge about following stimuli within this small number of trials: Peak differences for harder versus softer stimuli in the recurring condition of Experiment 1 did not significantly differ between the first and the second half of each block of overall 24 trials, t(23) = −0.364, p = 0.719, suggesting a solid level of learning within 12 trials (mean initial peak force difference between harder and softer stimuli in first half: M = 0.91N , SEM = 0.67; mean initial peak force difference between harder and softer stimuli in second half: M = 1.21N, SEM = 0.64). Test and control condition each contained 96 trials (24 comparisons per stimulus pair), so that each participant performed 192 trials in total. Between every of the 12 blocks there was a break of 15 s to prevent fatigue. Before starting with the main trials, participants performed eight practice trials (no prior information). The experiment was conducted in one session of, on average, 2 hr duration. 

Data analysis

Again, we analyzed the initial peak forces per trial. We compared individual averages separately for the two groups (interference group, implicit-only group) using two ANOVAs with the two within-participant variables Prior Information (available, control), and Compliance Category (softer, harder). We expected a significant interaction between the factors Prior Information and Compliance Category in the implicit-only group, with a larger hard-minus-soft difference of initial peak force when Prior Information is available as compared to without prior information in the control group. We did not expect this interaction in the interference group. We also calculated planned t tests between harder and softer categories within each Prior Information condition for each group. In the implicit-only group we expected higher initial peak forces for harder compared to softer stimuli in the blocked condition, and no effect in the control condition. In the interference group we did not expect an effect in either Prior Information condition. We used G*Power (Faul, Erdfelder, Buchner, & Lang, 2009) to estimate the number of participants needed to get a good power for tests within and between groups. Based on the within participant effect size of 1 observed in Experiment 1 and in Lezkan and Drewing (2015) for our sample size of N = 12 the estimated power was 0.95 (one tailed) for detecting an effect of Compliance Category within each group, which corresponds to a β-error of falsely accepting the null hypothesis (no considerable effect) of 5%. In contrast, to find a medium-sized between-groups effect (0.5) with equally good power, we would have needed an unrealistic total number of 176 participants. Hence, we decided not to calculate between-groups effects, while having an acceptable level of confidence for accepting the null hypothesis of within-group comparisons. 

Results

In the ANOVA of peak forces for the implicit-only group (Figure 8), we found a significant main effect of the variable Compliance Category, F(1, 11) = 10.606, p = 0.008, and a significant interaction Compliance Category × Prior Information, F(1, 11) = 4.195 p = 0.033 (one-sided), indicating as expected that initial peak force was higher for harder than for softer stimuli in particular if participants had obtained (implicit) prior information on the compliance category. Additionally, the corresponding Bayes factor (BF) favored the alternative hypothesis, BF = 2.35. Planned t tests confirm the hard-soft difference (2.47 N on average) for the condition with prior information, t(11) = 1.967, p = 0.037 (one-sided), and show as expected no significant difference in the control condition, t(11) = 0.831, p = 0.424. The overall main effect of Prior Information was not significant for the implicit-only group, F(1, 11) = 2.678, p = 0.130. In the ANOVA for the interference group, no significant effect was found: Compliance Category, F(1, 11) = 2.643, p = 0.132; Prior Information, F(1, 11) = 1.300, p = 0.278; and interaction, F(1, 11) = 0.656, p = 0.435. Also in the planned t tests, initial peak forces for harder compared to softer stimuli did not differ, neither with explicit prior knowledge, t(11) = 1.129, p = 0.283, nor in the control condition, t(11) = 0.704, p = 0.496. Because we expected no effect for the interaction, we calculated the corresponding Bayes factor, which favored the null hypothesis (i.e., peak forces for softer and harder stimuli do not differ for explicit or no prior information), BF = 0.22. 

Figure 8

 

Average initial peak force (N) and ipsative standard error of the mean (SEM; Loftus & Masson, 1994) as a function of compliance category and prior Information.

On average participants performed 12.0 indentations per trial, and changed 4.4 times between stimuli. Individual performance ranged from 63% to 89%. In the implicit-only group participants gave 75% correct responses in the control condition and 80% in the test condition. In the interference group, participants gave 73% correct responses in the control condition and 75% in the test condition. Please note that in both groups participants performed better for softer stimuli as compared to harder stimuli: implicit-only group: t(11) = 4.008, p = 0.002, harder stimuli = 73%, softer stimuli = 82%; interference group: t(11) = 3.532, p = 0.005, harder stimuli = 70%, softer stimuli = 79%. 

Discussion: Experiment 2

As expected, we found that implicit information alone resulted in the use of higher peak force for harder as compared to softer stimuli, whereas implicit plus explicit information did not result in a corresponding adaptation. We conclude that exploratory behavior in haptic perception can be improved by implicit information on the to-be-explored stimuli, whereas explicit knowledge interferes with such adaptation. 

The result of initial peak force adaptation given implicit prior information highlights the usage of implicit knowledge in haptic exploration. Further, the result that adding explicit information eliminates the adaptation effect leading to no changes in initial peak force can well explain the findings for the semantic and visual conditions of Experiment 1. In both conditions prior information was given explicitly, as in the interference group of Experiment 2. We found no adaptation of initial peak forces in the semantic condition of Experiment 1, as in the interference group of Experiment 2, and we found an effect in the opposite direction in the visual condition. These results can be partly explained with theories arguing that task control depends on the type of knowledge that is used (Anderson, 1993; Fisk et al., 1987). In our case, explicit information might have triggered a consciously controlled execution of the exploration task, differing from an automatic execution induced by implicit information. The explicit processing of a task that is usually implicit can reduce performance and lead to ineffective behavior (Masters, 1992). The present finding that implicit prior knowledge improves motor adaptation, but implicit plus explicit prior knowledge does not, is consistent with our suggestion that explicit information interferes with the implicit automatic exploration process, supporting a consciously controlled explicit process. Regarding the opposite effect in the visual condition of Experiment 1, we speculate that the closer connection between the haptic and visual sense as compared to the semantic condition (Newell et al., 2001) may have led to higher interference and a higher effect on the exploration behavior. However, to confirm this speculation, further experiments need to be done. 

Finally, although we had carefully matched discrimination performance for hard and soft stimuli in a pilot experiment, in the main experiment, soft stimuli were better discriminated. Given that smaller compliance differences between stimuli (or more difficult discrimination tasks) have been previously reported to lead to the use of higher peak forces (Kaim & Drewing, 2011), higher peak forces for the harder as compared to the softer set may be alternatively explained by differences in task difficulty. Note first that this would not affect our main conclusions on the usage of explicit and implicit prior information, because such prior information on task difficulty would have been available or not available in exactly the same conditions as the prior information on compliance category. In addition, we tested for a correlation between the difference in performance and the difference in initial peak force used for softer as compared to harder stimuli during exploration in the test condition of the implicit group. If there was a relevant influence of the difficulty, participants with higher differences in performance between compliance categories should also have a systematically more pronounced force difference. However, we did not find a significant correlation, r = 0.239, p = 0.454. We hence assume that we mainly replicated the previously observed effects of predicted compliance category on peak force (Experiment 1; Kaim & Drewing, 2011; Lezkan & Drewing, 2015). 

General discussion and conclusions

We investigated how prior information is integrated during a haptic task and how it changes exploration behavior. In particular, we examined how different types of prior knowledge on compliance category, presented through different channels (recurring compliance, visual, semantic) and inducing different knowledge types (implicit, explicit), influence the haptic exploration in softness discrimination. In Experiment 1 we were able to replicate previous findings (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). Participants used more initial peak force when exploring harder stimuli as compared to softer ones when prior information was given through recurring presentation of stimuli with similar compliance. However, participants showed no adaptation when prior information was given semantically, and an opposite effect with visual prior information. We wondered whether the unexpected results with semantic and visual information could be due to an interference of the explicit prior information on the implicit exploration process. Results of Experiment 2 corroborated this hypothesis: Implicit information alone led to an adaptation of initial peak forces to compliance category similar to the previous experiments. In contrast, participants who received additional explicit prior information showed on average no adaptation of initial peak forces when exploring stimuli. We conclude that the integration of prior knowledge in the implicit exploration process is highly dependent on the induced knowledge type. Adaptation seems to be only possible when implicit prior knowledge is induced but is not possible with explicit knowledge. 

When considering the use of different knowledge types, it should be noted that our definition of implicit and explicit knowledge is very closely connected to controlled and automatic information processing theory in tasks (Fisk et al., 1987). Following the theory, more explicit previous information could lead to more controlled and slower processing of task relevant information and motor commands. Therefore, explicit information could impair behavior in usually highly trained and rapidly processed motor tasks such as haptic exploration. Implicit information, by contrast, could lead to peak force adaptation and support the automatic and faster processing of information. Participants in an fMRI study (Pascual-Leone, Grafman, & Hallett, 1994) showed a different cortical pattern of activity in motor cortex when performing the same task with either implicit or explicit knowledge, highlighting a differential processing of implicit and explicit information, as described in Fisk et al. (1987). However, it should also be noted that the theory of automatic and controlled processing was built and tested with visual search tasks, rather than haptic discrimination (Schmidt & Lee, 2011). 

The observed force adaptation in implicit conditions should also have led to improved performance in gathering sensory information from the first indentation(s). This effect has been clearly demonstrated in a previous study, where force was varied in an otherwise highly constrained exploration (Kaim & Drewing, 2011). However, the present experiments were not designed to isolate the influence of specific exploratory parameters on overall performance. To keep the task as natural as possible, we did not constrain the number of later indentations, the number of changes between stimuli, or the forces used in later indentations. Participants could freely explore and strategically compensate (or overcompensate) for any early lack of information in any condition. That is, we cannot draw any specific conclusions from the performances observed in our study, and hence, also did not systematically compare performance data between conditions. 

We further argued that the visual information presented in the first experiment functioned as explicit information. However, in contrast to the information we presented during the experiment, it might be the case that during natural exploration initially explicit visual information is—after sufficient learning—accessed implicitly. Following (Anderson, 1983, 1993; Masters, 1992) new tasks are often executed in a controlled and explicit way to begin with. Once a task is learned well enough, knowledge can become implicit, so humans can execute the task and access relevant information without conscious thought. This could also be true for information used in haptic exploration. It seems intuitively obvious that we do not consciously think of all our past experiences whenever we judge the softness of an object in daily life, which would take a lot of time and energy (Fisk et al., 1987). Therefore, more natural visual prior information could be integrated unconsciously in natural exploration behavior (cf. Lederman & Klatzky, 1987; Masters, 1992; Sun et al., 2005). Such integration might, however, not be possible when decisive prior information about an unknown object is given explicitly directly before exploring. This argument leads to the question of how visual prior knowledge in the laboratory can represent prior knowledge in natural situations. We would expect that prior knowledge is not often given explicitly only immediately preceding exploration. Because people seem not to think explicitly about the properties before interacting with objects, explicit prior information might be exceptional in natural situations. To further test our idea that implicit prior information supports exploratory behavior whereas explicit information interferes with it, one could explore motor adaptation with different implicit sequences of hard and soft pairs. We know that simple motor sequences, for instances responding with different keyboard keys in a recurring sequence, can be implicitly learned (Fisk et al., 1987). If the adaptation of initial peak forces in softness discrimination tasks is a process that is based on implicit information, humans should also adapt to compliance category when the sequence of compliance category follows a simple pattern, that is, able to induce implicit knowledge of the upcoming stimuli. 

We explain the unexpected effect that with visual prior information participants used more force for softer than for harder stimuli by a strong interference of explicit visual information with implicit exploration behavior. Alternatively, one could speculate that participants tried to imitate the movement of the probe that they had seen in the video sequence, because the probe indented the stimulus deeper when indicating a soft stimulus. If participants would mimic the behavior of the probe, they would use more force for softer than for harder stimuli. However, before starting the experiment, participants were told that the video sequences are presented to give information about the stimuli and do not contain any instructions on the task. Because of this clarification it is not immediately obvious why participants would imitate the behavior of the probe. Note that Paulun et al. (2017) had already shown that participants gather softness information from our displayed visual stimuli in the intended manner, and also Cellini, Kaim, and Drewing (2013) found that humans are able to perceive softness from vision only, using stimuli comparable to ours. Still, from the available data, we cannot unequivocally decide if the unexpected effect found in the visual condition is due to a stronger interference, or to participants mimicking the probe. However, in both cases we can consider the prior visual information to be explicit information that interferes with the implicit exploration behavior. 

Furthermore, finding such specific adaptation effects, as we did, indicates a very complex and differentiated underlying process, in which prior information is integrated. Initially we hypothesized that any prior information might be integrated in the same way, leading to comparable influences on the haptic exploration process. This model is comparable to the Weak Observer model for depth cue integration, described by Landy, Maloney, Johnston, and Young (1994), in that both models assume that different inputs are first transformed into a shared format and then integrated by a weighted averaging process. In our case this is prior information gathered through different channels and sensory information (Lezkan & Drewing, 2018). The input format would need to be converted to common units in order to allow for a weighted averaging. The weights of different prior information might depend on many factors. We think that in natural situations, the quality of prior information often differs, for example due to differences in the specificity of information, memory decay of older information, relevance of the information in the present situation, or simply lack of attention. The Weak Observer model could explain the differences between the semantic and the recurring prior information condition. Because the semantic information might be less specific than information gathered out of repeated presentations, it might not be sufficient to lead to force adaptation. Results from the visual condition, however, cannot be explained by the model. Because the visual prior information seems to be very precise and relevant, it should have a high weight when integrating it, allowing for a good adaptation to the task. We did though not find a highly adaptive, but an opposite maladaptive effect on initial peak forces when showing visual prior information. The model also cannot explain the findings of Experiment 2. The availability of two sources of prior information should result in an averaged weighting of both, and thus better adaptation, but we found less adaptation when both implicit and explicit information was presented. Overall, we therefore conclude that our theory of the influence of explicit knowledge on the implicit exploration process can explain our data better than alternative explanations. 

Finally, in both experiments, implicit prediction of harder stimuli led to higher peak forces in the first indentation, which is in line with our expectations and with previous results (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). However, the hard-soft difference in the initial peak forces observed in Experiment 1 was smaller (1.0 N) than in previous studies (about 4.0 N in Kaim & Drewing, 2011; about 3.8 N for soft vs. hard stimuli in Lezkan & Drewing, 2015), and smaller than in Experiment 2 (about 2.5 N). One may speculate that participants may not have been able to reach a solid level of implicit knowledge/learning in Experiment 1 (Masters, 1992) because the number of successive trials with the same compliance category was relatively small (48 compared to 192 in Lezkan & Drewing, 2015). However, the number of trials in Experiment 2 was even smaller (16), but the force adjustment more pronounced. Alternatively, the smaller hard-soft effects in the present implicit condition could be due to differences in compliance levels. Stimuli in Experiment 1 had compliances of 0.45 mm/N and 0.95 mm/N, whereas stimuli in the harder conditions in the previous studies and in Experiment 2 were harder, and the difference between compliance categories was bigger (0.14 mm/N and 0.74 mm/N in Lezkan & Drewing, 2015; 0.15 mm/N and 1.24 mm/N in Kaim & Drewing, 2011; 0.15 mm/N and 0.95 mm/N in Experiment 2). We had used more compliant stimuli in the harder category of Experiment 1 to achieve a good match to the visual compliance display. Still, less difference between the harder and the softer category should lead to a smaller difference in adaptation (cf. Lezkan & Drewing, 2015). Additionally, the overall high level of compliance might have caused less difference in peak forces between harder and softer stimuli, due to a floor effect, given that humans seem always to use a certain minimum of initial peak force when exploring softer, or unknown stimuli (Kaim & Drewing, 2011; Lezkan & Drewing, 2015). Finally, one may ask why people use the same low force for unknown stimuli that they use for the soft stimuli, and higher force for harder stimuli—and not the other way around. We speculate that this is an efficient strategy. When no prior information is given, the use of small forces avoids wasting effort in a process in which the compliance of the explored stimulus is yet unknown. However, this speculation remains to be tested. 

Taken together, we conclude that prior knowledge does not always lead to movement adjustments in softness discrimination tasks. Depending on the type of information in which prior knowledge is presented, adaptation in exploration behavior can be supported (implicit information) or inhibited (explicit information). 

Acknowledgments

We thank Tamara Dobrjanski, Luisa Stricker, Elena Führer, Alexander Nieß, Judith Decker, and Claire Weyel for their help in producing the stimuli and collecting the data. This research was supported by German Research Foundation (DFG; CRC/TRR135, A05, C01). Data of both experiments presented here are available at https://doi.org/10.5281/zenodo.2549487. 

Commercial relationships: none. 

Corresponding author: Aaron C. Zoeller 

Email: . 

Address: Department of General Psychology, Giessen University, Gießen, Germany.