Journal Information
Vol. 100. Issue 2.
Pages 143-148 (March - April 2024)
Share
Share
Download PDF
More article options
Visits
922
Vol. 100. Issue 2.
Pages 143-148 (March - April 2024)
Original article
Full text access
Implementing palliative care, based on family-centered care, in a highly complex neonatal unit
Visits
922
Ana Morillo Palomo
Corresponding author
ana.morillo@sjd.es

Corresponding author.
, Jordi Clotet Caba, Marta Camprubí Camprubí, Elena Blanco Díez, Joel Silla Gil, Ana Riverola de Veciana
Sant Joan de Déu Barcelona Children's Hospital, Neonatal Intensive Care Unit, Barcelona, Spain
This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (1)
Tables (3)
Table 1. Epidemiological and clinical variables.
Table 2. Main condition for redirection of care.
Table 3. End-of-life support items.
Show moreShow less
Abstract
Objective

To describe the causes and circumstances of neonatal mortality and determine whether the implementation of a palliative care protocol has improved the quality of end-of-life care.

Methods

A retrospective observational study including all patient mortalities between January 2009 and December 2019. Cause of death and characteristics of support during the dying process were collected. Two periods, before and after the implementation of a palliative care protocol, were compared.

Results

There were 344 deaths. Congenital malformations were the most frequent cause of death (45.6 %). Most patients died after the transition to palliative care (74.4 %). The most frequently cited criteria for initiating transition of care was poor neurocognitive prognosis (47.2 %). Parents accompanied their children in the dying process in 72 % of cases. Twenty-three percent of patients died outside the Neonatal Intensive Care Unit after being transferred to a private room to enhance family intimacy. After the addition of the palliative care protocol, statistically significant differences were observed in the support and patient experience during the dying process.

Conclusions

The most frequent causes of death were severe congenital malformations. Most patients died accompanied by their parents after the transition to palliative care. The implementation of a palliative care protocol helped to improve the family-centered end-of-life care.

Keywords:
Neonatal death
Neonatal intensive care unit
Family-centered care
Redirection of care
Neonatal palliative care
Full Text
Introduction

Most deaths in the pediatric age group occur during the neonatal period and nearly all occur in neonatal intensive care units (NICUs).1 Despite that, it is important to highlight that the practice of palliative care is not always prevailing in NICUs.

Due to technological advances and perinatal care improvements, as well as higher quality treatment and care of children with disabilities, the infant mortality rate has declined dramatically in the past century.2 However, it is not always appropriate to do everything technically possible. In certain circumstances, intensive treatment should be replaced by palliative care in order to alleviate or prevent the suffering of newborns and their families.3 Thus, a large proportion of neonatal deaths occur after a decision to no longer initiate life-sustaining treatment is made or after the transition to palliative care.4

Palliative care in the NICU should focus not only on ensuring the best quality of life for the baby but also on psychosocial and spiritual support for the child's family.5 Some centers have already introduced palliative care in the prenatal period, hospice care, when a life-limiting or life-threatening disorder is diagnosed during gestation.6 Perinatal palliative care addresses decision-making and bereavement support for parents during pregnancy as well as birth planning and postnatal infant support.7

Individualized family-centered developmental care (IFCDC) is a philosophy of care, applied during the neonatal period, which aims to reduce the impact on newborns and their families. Admission of a sick or preterm newborn to the neonatal unit can be an extremely stressful experience for both, the infant and family and, in many cases, requires a long stay in the NICU.

The key focus of IFCDC is the newborn and the family, with the family's involvement being fundamental in the direct care of the child. NICUs that incorporate IFCDC represent a paradigm of neonatal palliative care (NPC) of the highest quality, given the comprehensive and holistic approach to newborns and their families.8,9

The objectives of this study are to describe the main causes of death and modes of death of patients admitted to our NICU. The authors also aim to assess whether the implementation of an NPC protocol has improved the end-of-life care of our newborns.

MethodsStudy design and population

A retrospective observational study was conducted in the NICU of Sant Joan de Déu Barcelona Children's Hospital in Esplugues de Llobregat (Barcelona Prov.). This is a tertiary-level university hospital and referral center for a substantial part of the population of Barcelona and Catalonia.

The NICU at Sant Joan de Déu Barcelona Children's Hospital carries out complex, highly specialized procedures. Despite a huge complex technical environment, the unit has remained committed to the implementation of the IFCDC philosophy for several years. The Newborn Individualized Developmental Care and Assessment Program (NIDCAP) was adopted many years ago and the NICU later became a NIDCAP training center. During the study period, the NICU was divided into wards, accommodating several inpatients each (6–10 patients per ward).

All patients who died in the Neonatal Unit from January 1, 2009 to December 31, 2019 were included.

For data analysis, two time periods were compared. Period 1 included patients who died from 2009 to 2015 and Period 2 those who died from 2016 to 2019.

The two periods were defined based on the following circumstances:

In 2012, palliative care was started in a regulated manner in the NICU. A palliative care protocol was developed, including a description of criteria for recommending transition of care, support throughout the care process (clinical controls and care, monitoring, changes in treatment, analgesia and sedation, environment and attention to the family), steps to follow after death, and follow-up for the family after discharge, with an annex on referrals to the Pediatric Palliative Care Unit.

From 2013 to 2015, palliative care training was provided to all NICU staff in order to facilitate the implementation of the new protocol.

Variables

Data collection was performed retrospectively, with a medical record review after patient death.

The following variables were collected:

  • -

    Epidemiological variables (gestational age, weight, origin, gender, age at death), clinical variables (main pathology, cause of death), acceptance or refusal of necropsy, and valve and/or tissue donation.

  • -

    Variables related to care plan during death (application or not to transition of care, transition of care criteria).

  • -

    Variables related to support at the end of life: sedation and analgesia practices during the dying process, place of death, photography by parents, rituals or ceremonies according to the cultural or religious beliefs of patient families, and creation and delivery of remembrance material. Additionally, referrals to the Pediatric Palliative Care Unit were also recorded.

Statistical analysis

The variables collected were analyzed using descriptive statistics (frequency distribution for categorical data, mean and standard deviation or median and interquartile range for continuous data). The chi-square test and Fisher's exact test were used to compare categorical variables, and the student's t-test or Mann-Whitney test were run for continuous variables. Statistical significance was deemed present in p-values less than 0.05. The data were analyzed with the IBM SPSS Statistics 17.0 statistical program.

Ethics statements

The retrospective study was approved by the Research Ethics Committee Fundació Sant Joan de Déu (I.C PIC-266-2) with a waiver of written consent.

ResultsPopulation description

Over the 11 years of study time, there were 38.203 deliveries in the studied center, 7061 admissions into the Neonatal Unit, 75.2 % of whom were inborn patients, and a total of 344 neonatal deaths.

Regarding general characteristics of the population, the mean gestational age (GA) was 34+1 weeks with a minimum of 23 and a maximum of 44 weeks (SD 5.63). In 70.1 % of deceased patients, the GA was greater than 32 weeks. The mean weight was 2.175 gs (SD: 1073.08) and the mean age at time of death was 15.2 days (SD: 21.74). Eighty-four percent of deceased patients were less than 28 days old. Necropsy was performed in 61.7 % of the patients who died after the transition of care, similar to 61.9 % of deceased patients overall. 3.2 % of families opted for valve donation in the study period. These general clinical and epidemiological characteristics of the two periods are shown in Table 1.

Table 1.

Epidemiological and clinical variables.

  Period 1 (n = 256)  Period 2 (n = 88)  p-value 
Gestational age (weeks)  34.09  34.08  0.994 
Birth weight (grams)  2165  2203  0.781 
Male gender (%)  58.2  57.9  0.967 
Redirection of care (%)  72.2  80.7  0.119 
Inborns (%)  58.9  60.2  0.286 
Necropsy (%)  66.6  50  *0.031 

Congenital malformations were the most frequent cause of death (45.9% of the deceased patients). Complex cardiac defects, diaphragmatic hernia, chromosomal disorders and/or polymalformative syndromes, and neurological malformations were the most commonly found anomalies. Complications related to prematurity represent 25% of the deceased patients, being severe central nervous system lesions (severe intraventricular bleeding with parenchymal extension) and severe necrotizing enterocolitis the complications. The third cause of death was severe HIE with 16.2% of deceased patients followed by infections (4.6 %).

Description of care transitions

Regarding circumstances during death, most patients (74.4 %) died after the transition to palliative care. The complications related to severe congenital malformation are the main condition for redirection of care, followed by prematurity complications and severe hypoxic-ischemic encephalopathy (HIE). Table 2 shows the transition of care according to the cause of death for both study periods.

Table 2.

Main condition for redirection of care.

  Period 1 (n = 185)  Period 2 (n = 71)  p-value 
Malformation  40 %  45 %  0.778
Complications due to prematurity  27 %  19.7 % 
Infection  4.3 %  4.2 % 
Severe HIE  23.3 %  18.4 % 
Other  5.4 %  12.7 % 

Poor neurocognitive prognosis, identified in 47.3 % of the deceased patients, was the most frequently cited criterion in the decision to transition patients to palliative care. Redirection of care involved withdrawal of life-sustaining treatment in 71.5 % of the deceased patients, while in 28.5 % no new treatments were added and cardiopulmonary resuscitation was not initiated, no differences between period 1 and period 2 were seen.

Description of end-of-life support

Among patients who died after the transition of care, 85.9 % received some type of medication (analgesia and/or sedation) during the dying process to ensure comfort.

In 50.4 % of cases, parents were asked if they would like to perform some type of religious ceremony or ritual before or after death. Of the 129 families inquired, 63 answered affirmatively.

Taking pictures of the baby or of the baby with the family, by parents or by healthcare professionals, was observed and recorded in the clinical history of 41.4 % of patients.

Regarding the creation of mementos, in 2010 the nursing team began to make handmade and personalized albums, for families. There is no clinical record of whether albums were given to parents from 2009 to 2016. From 2017 through 2019, all families whose children died in the Neonatal Unit, regardless of whether treatment was adapted to palliative care or not, were offered the creation of keepsakes and/or participated in the process.

Seventy-two percent of patients who died after palliative care were accompanied by their parents. Twenty-eight percent of patients died outside the NICU after being transferred to a private room in order to enhance family intimacy during the dying process (Figure 1).

Figure 1.

End-of-life support differences between period 1 and period 2. (A): Patients photography by families over time (%). A progressive increase in photography was found. (B): Place of death (%), In recent years, the authors observed a steady increase in the transfer of newborns and their families to single rooms.

(0.2MB).

In the eleven years of the study period, a total of 24 patients were referred to the Pediatric Palliative Care Unit. Of these 24 patients, 4 died in the neonatal wards while admitted to the studied unit and were evaluated in the group of 256 patients who received neonatal palliative care. The remaining patients were discharged from the hospital and subsequently died at home or in another ward of the hospital after readmission. It should be noted that the largest number of referrals to the Pediatric Palliative Care Unit was made from 2015 onwards as part of the protocol established for some patients.

Description of end-of-life support between periods

As explained in the methodology, the second period was defined due to the significant change in the implementation and management of palliative care that occurred at the beginning of this period. With the progressive implementation of family-centered developmental care in the neonatal unit and training in palliative care, there was a concomitant and continuous improvement in the support of patients during end-of-life.

Differences between the two periods are shown in Table 3.

Table 3.

End-of-life support items.

Period  P1 (n = 185)  P2 (n = 71)  p-value 
Photography  75 %  91.7 %  0.013* 
Death in a private room  15 %  48.6 %  0.001* 
Parental presence  90.9 %  98.5 %  0.039* 
Analgesia use  97.5 %  96.97 %  0.82 
Offer of baptism or ritual  91.7 %  86.6 %  0.33 
Performance of ritual  48.8 %  34.4 %  0.082 

*p < 0.05. The presence of parents during the dying process, photography, and place of death were different with statistical significance between the two periods. There was an increase in the number of babies transferred from the NICU to single rooms to favor the intimacy of the baby's family in the end-of-life period (p = 0.0001). No statistical difference in the performance of rituals was detected between the two periods, though a trend towards increased rituals was observed in the second period.

Discussion

In this study, the authors have described the main causes of death, the end-of-life care plan, and the strategies of support in the dying process in the NICU. The authors have also described some improvements in the end-of-life care of the studied patients between the two periods compared. Despite the increasing presence of pediatric palliative care in the evaluated country, to our knowledge, this is the first study analyzing the causes of death, modes of dying, and end-of-life support in a NICU in Spain.

Unlike other studies, which reported the main cause of death as prematurity and its complications,10,11,12 the gestational age for those that died in the population was relatively high at 34.1 weeks, with congenital malformations as the main cause of death. This is likely due to the degree of specialization and high level of care provided by the present study's center. These results are in accordance with those published in a study conducted in an Australian tertiary-level center, which found the main cause of neonatal mortality to be congenital malformations, followed by prematurity. The percentages presented by these authors are very similar to those of the present cohort, in line with those reported by other level III units.13,14 These data suggest that the cause of death depends on the type of patients involved as well as the level of care and resources offered by the neonatal unit.

According to Hellmann et al.,1 most published studies on death in neonatal units do not specify which criteria are followed in decisions to transition to palliative care. In the studied population, the severity of neurocognitive prognosis, followed by the inevitability of death within a short time, were the main criteria for deciding on the transition of care.

In this study, most deaths in the neonatal unit (74.4 %) occurred after the decision to transition care to palliative care. These results are in accordance with those reported by Wilkinson in his review of death in NICUs, suggesting that redirection of care in NICUs has remained stable over the last few years.15,16 In the present cohort, transition of care mainly involved the withdrawal of life-sustaining treatment. This data is consistent with what has been previously published.17,18 Another retrospective study on the dying process in the pediatric age group reported that the NICU is the place in the hospital where most patients die after withdrawal of life-sustaining treatment.

Several guidelines have been published for how to proceed with respect to the transition of care.19 The use of pain medication or sedation during the dying process to ensure infant comfort and prevent pain, discomfort, or agitation is widely supported in medical literature.14,20

Though most patients die accompanied by their parents in the studied unit, it is important to note that in recent years there has been an increasing number of patients who have been moved with their families to a single room in order to improve end-of-life support and enhance family intimacy during the dying process. It is essential to consider these fundamental decisions, to enhance them and to actively promote them. It is widely documented that these practices help families deal with their subsequent grief.21,22

Similarly, creating tangible memories, such as photographs and albums, and encouraging non-tangible memories, such as being physically present, holding, and cuddling babies are fundamental aspects with healing effects for parents who go through this experience.23–25 There is evidence that these memories will provide reassurance of the existence of their child and validate their role as parents of the deceased child.26 In addition, these practices are indicators of the quality of end-of-life care for newborns.14 Over time, however, studies have shown that mental health morbidity is higher in mothers who are not been able to be with their children in the last moments of their lives compared to mothers who do spend time with them and create memories together.27 Therefore, parents should be supported and encouraged to create memories of their children and to care for them during their lives, no matter how brief.

A point not yet explored in this study is the social and emotional support for bereavement after death. Though the authors typically have several meetings with parents after their child's death, this practice is not protocolized. Cortezzo et al. published a descriptive study with interviews of medical professionals and families experiencing end-of-life care in a neonatal unit. Families emphasized the need for post-death follow-up and bereavement support after this process.22 It is clear that the support of families after the baby's death and beyond hospital admission is crucial and, if possible, should be performed by healthcare professionals.

Another key recommendation highlighted in the literature is the need for training in palliative care and perinatal bereavement for medical professionals working in delivery rooms, obstetrician's offices, and NICUs.7,22,25 In the present study's center, training was provided after the development of the palliative care protocol. These results show that there was an improvement in providing adequate quality care to patients who received palliative care in Period 2, supporting the hypothesis that training specific to palliative care is essential to provide quality, empathetic, and compassionate care to families of such patients.

It is imperative that clinical guidelines for end-of-life support be created.28,29 In a study published in 2018, NICU professionals in the USA and Canada were surveyed to evaluate the end-of-life care of newborns. The results obtained illuminated the absence of formal end-of-life care guidelines at many institutions. Furthermore, for institutions that do have guidelines, many lack important palliative care aspects such as symptom management and caregiver support.30 These results reinforce the importance of implementing and protocolizing training in perinatal palliative care in NICUs.

The present study has some limitations. First, as with most retrospective studies, data collection can be cumbersome or difficult. An awareness of and training in palliative care likely improved the efficiency of data collection, especially in the second period, suggesting a possible collection bias. Likewise, all results were generated from one hospital meaning their generalizability could be limited, given group dynamics of each center may be different, particularly considering the subjective nature of studying palliative care. Though incorporating palliative care protocols is vital in all neonatal care settings, some of the specific patient characteristics reported may be more applicable to other tertiary care centers. Last, but not least, the authors have not explored the social and emotional support for post-death bereavement and have identified this as both a subject of future investigation and an area of improvement for the present study's center.

This data provides evidence that congenital malformations were the most frequent cause of death and that most patients die after transition to palliative care.

In addition, this study explores differences in the management of end-of-life support between two periods and illuminates changes in the families' experience after protocolizing and increasing specific training for palliative care.

Being a family and developmental care center has facilitated a more holistic view of end-of-life care and, consequently, the implementation of the palliative care protocol during the years under study.

A family-centered approach to death means offering families an adequate and intimate place where their child may die with dignity and in the company of loved ones with the constant and respectful support of medical professionals.

References
[1]
J. Hellmann, R. Knighton, S.K. Lee, P.S. Shah, Canadian Neonatal Network End of Life Study Group.
Neonatal deaths: prospective exploration of the causes and process of end-of-life decisions.
Arch Dis Child Fetal Neonatal Ed, 101 (2016), pp. F102-F107
[2]
Ministerio de Sanidad. Patrones de Mortalidad en España.
Información y Estadísticas Sanitarias 2017.
Ministerio de Sanidad, (2017),
[3]
V. Larcher.
Ethical considerations in neonatal end-of-life care.
Semin Fetal Neonatal Med, 18 (2013), pp. 105-110
[4]
M. Kilcullen, S. Ireland.
Palliative care in the neonatal unit: neonatal nursing staff perceptions of facilitators and barriers in a regional tertiary nursery.
BMC Palliat Care, 16 (2017), pp. 32
[5]
D. Discenza.
Palliative care in the NICU and why education matters.
Neonatal Netw, 36 (2017), pp. 48-49
[6]
A. Balaguer, A. Martín-Ancel, D. Ortigoza-Escobar, J. Escribano, J. Argemi.
The model of Palliative Care in the perinatal setting: a review of the literature.
BMC Pediatr, 12 (2012), pp. 25
[7]
C. Wool, D. Côté-Arsenault, B. Perry Black, E. Denney-Koelsch, S. Kim, K Kavanaugh.
Provision of services in perinatal palliative care: a multicenter survey in the United States.
J Palliat Med, 19 (2016), pp. 279-285
[8]
C. Kenner, J. Press, D. Ryan.
Recommendations for palliative and bereavement care in the NICU: a family-centered integrative approach.
J Perinatol, 35 (2015), pp. S19-S23
[9]
N. Younge, P.B. Smith, R.N. Goldberg, D.H. Brandon, C. Simmons, C.M. Cotten, M. Bidegain.
Impact of a palliative care program on end-of-life care in a neonatal intensive care unit.
J Perinatol, 35 (2015), pp. 218-222
[10]
J. Jacob, M. Kamitsuka, R.H. Clark, A.S. Kelleher, A.R. Spitzer.
Etiologies of NICU deaths.
Pediatrics, 135 (2015), pp. e59-e65
[11]
S. Eventov-Friedman, H. Kanevsky, B. Bar-Oz.
Neonatal end-of-life care: a single-center NICU experience in Israel over a decade.
Pediatrics, 131 (2013), pp. e1889-e1896
[12]
T.M. Berger, A. Hofer.
Causes and circumstances of neonatal deaths in 108 consecutive cases over a 10-year period at the Children's Hospital of Lucerne, Switzerland.
Neonatology, 95 (2009), pp. 157-163
[13]
M.C. Michel, T.T. Colaizy, J.M. Klein, J.L. Segar, E.F. Bell.
Causes and circumstances of death in a neonatal unit over 20 years.
Pediatr Res, 83 (2018), pp. 829-833
[14]
D. Gilmour, M.W. Davies, A.R. Herbert.
Adequacy of palliative care in a single tertiary neonatal unit.
J Paediatr Child Health, 53 (2017), pp. 136-144
[15]
D.J. Wilkinson, J.J. Fitzsimons, P.A. Dargaville, N.T. Campbell, P.M. Loughnan, P.N. McDougall, et al.
Death in the neonatal intensive care unit: changing patterns of end of life care over two decades.
Arch Dis Child Fetal Neonatal Ed, 91 (2006), pp. F268-F271
[16]
L. Barton, J.E. Hodgman.
The contribution of withholding or withdrawing care to newborn mortality.
Pediatrics, 116 (2005), pp. 1487-1491
[17]
C.M. Hagen, T.W. Hansen.
Deaths in a neonatal intensive care unit: a 10-year perspective.
Pediatr Crit Care Med, 5 (2004), pp. 463-468
[18]
K. Zimmermann, E. Cignacco, S. Engberg, A.S. Ramelet, N. von der Weid, K. Eskola, et al.
Patterns of paediatric end-of-life care: a chart review across different care settings in Switzerland.
BMC Pediatr, 18 (2018), pp. 67
[19]
F.J. Cambra, M. Iriondo, M.A. Andreu, R. Badosa, R. Escobar, S. Gabaldón, et al.
Limitación Terapéutica En Pediatría.
Ediciones Sant Juan de Dios Campus Docent, (2015),
[20]
D.E. Cortezzo, M. Meyer.
Neonatal end-of-life symptom management.
Front Pediatr, 8 (2020),
[21]
L. Caeymaex, C. Jousselme, C. Vasilescu, C. Danan, B. Falissard, M.M. Bourrat, et al.
Perceived role in end-of-life decision making in the NICU affects long-term parental grief response.
Arch Dis Child Fetal Neonatal Ed, 98 (2013), pp. F26-F31
[22]
D.E. Cortezzo, M.R. Sanders, E.A. Brownell, K. Moss.
End-of-life care in the neonatal intensive care unit: experiences of staff and parents.
Am J Perinatol, 32 (2015), pp. 713-724
[23]
E. Aiyelaagbe, R.E. Scott, V. Holmes, E. Lane, A.E. Heazell.
Assessing the quality of bereavement care after perinatal death: development and piloting of a questionnaire to assess parents' experiences.
J Obstet Gynaecol, 37 (2017), pp. 931-936
[24]
C. Blood, J. Cacciatore.
Best practice in bereavement photography after perinatal death: qualitative analysis with 104 parents.
BMC Psychol, 2 (2014), pp. 15
[25]
L.D. LeDuff 3rd, W.T. Bradshaw, S.M Blake.
Transitional objects to faciliate grieving following perinatal loss.
Adv Neonatal Care, 17 (2017), pp. 347-353
[26]
R. Thornton, P. Nicholson, L. Harms.
Creating evidence: findings from a grounded theory of memory-making in neonatal bereavement care in Australia.
J Pediatr Nurs, 53 (2020), pp. 29-35
[27]
K. Erlandsson, J. Warland, J. Cacciatore, I. Rådestad.
Seeing and holding a stillborn baby: mothers' feelings in relation to how their babies were presented to them after birth–findings from an online questionnaire.
Midwifery, 29 (2013), pp. 246-250
[28]
S. Haug, A. Dye, S. Durrani.
End-of-life care for neonates: assessing and addressing pain and distressing symptoms.
Front Pediatr, 8 (2020),
[29]
S. Uthaya, A. Mancini, C. Beardsley, D. Wood, R. Ranmal, N. Modi.
Managing palliation in the neonatal unit.
Arch Dis Child Fetal Neonatal Ed, 99 (2014), pp. F349-F352
[30]
S. Haug, S. Farooqi, C.G. Wilson, A. Hopper, G. Oei, B. Carter.
Survey on neonatal end-of-life comfort care guidelines across America.
J Pain Symptom Manage, 55 (2018), pp. 979-984
Copyright © 2023. Sociedade Brasileira de Pediatria
Idiomas
Jornal de Pediatria (English Edition)
Article options
Tools
en pt
Taxa de publicaçao Publication fee
Os artigos submetidos a partir de 1º de setembro de 2018, que forem aceitos para publicação no Jornal de Pediatria, estarão sujeitos a uma taxa para que tenham sua publicação garantida. O artigo aceito somente será publicado após a comprovação do pagamento da taxa de publicação. Ao submeterem o manuscrito a este jornal, os autores concordam com esses termos. A submissão dos manuscritos continua gratuita. Para mais informações, contate assessoria@jped.com.br. Articles submitted as of September 1, 2018, which are accepted for publication in the Jornal de Pediatria, will be subject to a fee to have their publication guaranteed. The accepted article will only be published after proof of the publication fee payment. By submitting the manuscript to this journal, the authors agree to these terms. Manuscript submission remains free of charge. For more information, contact assessoria@jped.com.br.
Cookies policy Política de cookies
To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here. Utilizamos cookies próprios e de terceiros para melhorar nossos serviços e mostrar publicidade relacionada às suas preferências, analisando seus hábitos de navegação. Se continuar a navegar, consideramos que aceita o seu uso. Você pode alterar a configuração ou obter mais informações aqui.