Design1 - Segregated Components

Our first design allows parameters to be replaced with float or dict values. The associated components are segregated into an attribute of the model, which allows users to directly interact with these objects after a model instance is created.

Here's an example with a ConcreteModel:

model3a = ConcreteModel()
model3a.p = Param(initialize=-1)
model3a.q = Param(initialize=-1, mutable=True)
model3a.r = Param([0,1], initialize={0:0, 1:1})
model3a.s = Param([0,1], initialize={0:0, 1:1}, mutable=True)
model3a.x = Var()
def obj_rule(model):
    if model3a.p > 0:
        return model3a.x
    else:
        return -1*model3a.x
model3a.obj = Objective()
instance3a = model3a.create()

This results in the following:

instance3a.p            This is a float value
instance3a.q            This is a Param object for q
instance3a.r            This is a dict value
instance3a.s            This is a Param object for s
instance3a.components.p   This is a Param object for p
instance3a.components.q   This is a Param object for q
instance3a.components.r   This is a Param object for r
instance3a.components.s   This is a Param object for s

The obj_rule function works as expected, since model3a.p is a float value. Similarly, model3a.r is a parameter dictionary. However, the mutable parameters q and s continue to be Param objects.

I believe that this design can be implemented such that the data used in immutable parameters is shared, thereby avoiding a significant memory penalty for this design.

Unfortunately, this design is not backwards compatible. Thus, a user who has leveraged the API of Param objects may need to rewrite code to reference the components attribute of the instance.

Design2 - Segregated Data

Our second design segregates parameters to be expressed in a ''data'' attribute of the model. Here's an example with a ConcreteModel:

model3b = ConcreteModel()
model3b.p = Param(initialize=-1)
model3b.q = Param(initialize=-1, mutable=True)
model3b.r = Param([0,1], initialize={0:0, 1:1})
model3b.s = Param([0,1], initialize={0:0, 1:1}, mutable=True)
model3b.x = Var()
def obj_rule(model):
    if model3b.data.p > 0:
        return model3b.x
    else:
        return -1*model3b.x
model3b.obj = Objective()
instance3b = model3b.create()

This results in the following:

instance3b.data.p           This is a float value
instance3b.data.q           This is a Param object for q
instance3b.data.r           This is a dict value
instance3b.data.s           This is a Param object for s
instance3b.p                This is a Param object for p
instance3b.q                This is a Param object for q
instance3b.r                This is a Param object for r
instance3b.s                This is a Param object for s

This design is backwards compatible, since the Python data types are created within a well-known attribute of the model. However, a user might still get confused by the fact that model.p is not a float. That is, this design does not force a semantic change on the concrete model instance that protects the user, which is the main goal of this change in Pyomo semantics.

I believe that this design can be implemented such that the data used in immutable parameters is shared, thereby avoiding a significant memory penalty for this design.

Immutable Set Data

While considering the generation of data for immutable parameters, it seems natural to do the same for set data. This would have the advantage of presenting set data to the user in a native Python format, for which complex manipulations can be performed. Consider the following example, which generalizes the example in Design1:

model4a = ConcreteModel()
model4a.A = Set(initialize=[0,1])
model4a.B = Set(initialize=[0,1], mutable=True)
model4a.p = Param(initialize=-1)
model4a.q = Param(initialize=-1, mutable=True)
model4a.r = Param(model4a.A, initialize={0:0, 1:1})
model4a.s = Param(model4a.B, initialize={0:0, 1:1}, mutable=True)
model4a.x = Var()
def obj_rule(model):
    if model4a.p > 0:
        return model4a.x
    else:
        return -1*model4a.x
model4a.obj = Objective()
instance4a = model4a.create()

This results in the following:

instance4a.A            This is a Python set object
instance4a.B            This is a Pyomo Set object for B
instance4a.p            This is a float value
instance4a.q            This is a Param object for q
instance4a.r            This is a dict value
instance4a.s            This is a Param object for s
instance4a.components.A   This is a Set object for A
instance4a.components.B   This is a Set object for B
instance4a.components.r   This is a Param object for r
instance4a.components.p   This is a Param object for p
instance4a.components.q   This is a Param object for q
instance4a.components.r   This is a Param object for r
instance4a.components.s   This is a Param object for s

This design has the advantage that immutable set objects are native Python objects. As with parameters, a user might naturally expect this to be the case. Although Pyomo Set objects mimic Python set objects in many ways, there are subtle differences that might confuse a user.

However, there is a major memory issue lurking in this use of immutable set objects. The set A is a Python set object. When this is passed as an argument to the Param constructor, Pyomo creates a Set object to represent this indexing set, and then initializes it with the value of A. Thus, the data in A is duplicated in the model!

By contrast, when parameter s is constructed, set B is passed to the Param constructor. Here, B is a Set object, which does not need to be duplicated.

Thus, this design both eliminates a source of subtle bugs and introduces another. Note, however, that since the model is concrete, the components are being generated as they are declared. Consequently, the following declaration for parameter r would avoid this performance issue:

model4a.r = Param(model4a.components.A, initialize={0:0, 1:1})

Note that this memory issue with immutable sets only arises in concrete models. In abstract models, the set object construction is delayed, which allows the parameter objects to be defined with Set objects. (I could provide an example here, but it wouldn't convey the dynamic nature of what is happening...)

Abstract Models

The previous designs used ConcreteModels since that simplified the presentation of the examples. These designs should also work well for AbstractModel objects. Although one might argue that we should treat abstract models differently, the unclear parameter semantics has impacted users of both concrete and abstract models.

However, if we make this change, then the construction of concrete model instances from abstract models will be a big change in Pyomo. This will result in the copying of the abstract model objects, rather than creating an instance 'in place', as is currently done within Pyomo. There will likely be a performance hit for doing this additional work. Still, this should be a modest impact for large models.

Recommendation

Based on this comparison, Design1 seems the best alternative. This design protects the user from an unexpected semantics for parameter objects.

It is unclear whether to generalize immutability to set data, given the potential for memory bloat with subsequent component declarations.